Introduction to Neural Networks: Bank Churn prediction

Problem Statement

Context

Businesses like banks which provide service have to worry about problem of 'Customer Churn' i.e. customers leaving and joining another service provider. It is important to understand which aspects of the service influence a customer's decision in this regard. Management can concentrate efforts on improvement of service, keeping in mind these priorities.

Objective

You as a Data scientist with the bank need to build a neural network based classifier that can determine whether a customer will leave the bank or not in the next 6 months.

Data Dictionary:

• CustomerId: Unique ID which is assigned to each customer
• Surname: Last name of the customer
• CreditScore: It defines the credit history of the customer.
• Geography: A customer’s location
• Gender: It defines the Gender of the customer

• Age: Age of the customer

• Tenure: Number of years for which the customer has been with the bank

• NumOfProducts: refers to the number of products that a customer has purchased through the bank.

• Balance: Account balance

• HasCrCard: It is a categorical variable which decides whether the customer has credit card or not.

• EstimatedSalary: Estimated salary

• isActiveMember: Is is a categorical variable which decides whether the customer is active member of the bank or not ( Active member in the sense, using bank products regularly, making transactions etc )

• Exited : whether or not the customer left the bank within six month. It can take two values 0=No ( Customer did not leave the bank ) 1=Yes ( Customer left the bank )

Steps or Approach to Study

Step 1: Importing the Libraries

To begin with, we need to import the necessary libraries and dependencies that we will be using throughout the project. These include pandas, NumPy, matplotlib, seaborn, scikit-learn, and TensorFlow

Step 2: Data Analysis & Visualization

Next, we need to analyze and visualize the dataset to gain insights into the dataset and understand the variables that may contribute to customer churn.

Step 3: Apply One Hot Encoding Technique on Gender & Geography Column

Since our dataset contains categorical variables like “Gender” and “Geography,” we need to convert them into numerical format for the ANN model to process. We apply the one-hot encoding technique to transform these categorical columns into binary columns, representing different categories

Step 4: Handle the Imbalanced Target Column and Balance It

Customer churn datasets often suffer from class imbalance, where the number of churners is significantly smaller than non-churners. To address this issue, we will balance the target column by upsampling the minority class. This ensures that the model does not get biased towards the majority class during training

Step 5: Splitting the Dataset into Training and Testing Sets

Before training our ANN model, we need to preprocess the data by separating the features and the target variable. we split the dataset into training and testing sets. The training set is used to train the model on historical data, while the testing set is used to evaluate the model’s performance on unseen data. Typically, around 70–80% of the data is used for training, and the remaining 20–30% is used for testing

Step 6: Feature Scaling - Sklearn Standard Scaler Technique

Feature scaling is an essential preprocessing step for ANN models. It standardizes the range of input features, ensuring that they have similar scales. This step prevents certain features from dominating others during the training process. We apply feature scaling using techniques such as Sklearn Standard Scaler Technique for Standardized the Input Features Values.

Step 7: Build the Artificial Neural Network (ANN) Model

In this Step, Now it’s time to construct the architecture of the ANN model using the Keras library, which is a high-level API for building neural networks in TensorFlow. The model consists of multiple layers of interconnected neurons, including input, hidden, and output layers. We define the number of neurons in each layer, activation functions, and other parameters. The choice of the model architecture depends on the complexity of the problem and the available computational resources.

Step 8: Training the ANN Model

Once the model architecture is defined, we train the ANN model using the training dataset. During training, the model adjusts its internal parameters (weights and biases) to minimize the difference between the predicted churn outcomes and the actual outcomes. We specify the number of epochs (iterations) and the batch size for training. Monitoring metrics such as accuracy, loss, and validation metrics help assess the model’s performance during training.

Step 9: Visualize the Loss & Accuracy of ANN Model

Finally, we can visualize the model’s performance by plotting the training and validation accuracy and loss over the epochs

Step 10: Evaluating the ANN Model & Predicting Churn for New Customers

After training, we evaluate the performance of the ANN model using the testing dataset. We calculate various evaluation metrics such as accuracy, precision, recall, and F1 score to assess how well the model predicts customer churn.

Step 11: Predict the Churn and generate prediction score

Utilize the trained ANN model to predict churn for new customers by inputting their characteristics and behaviors into the model. The model generates a churn prediction score or probability, indicating the likelihood of the customer churning. Businesses can then take appropriate actions, such as targeted retention campaigns or personalized offers, to mitigate churn risk for these customers.

Step 12: Find the importance Feature of the Model

Use SHAP to look for the important feature in the model to be able to offer appropriate, business profesional insight.

Step 1 - Importing necessary libraries

# Library to suppress warnings or deprecation notes

# To help with reading and manipulating data
import pandas as pd
import numpy as np

# To help with data visualization
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
sns.set_style('darkgrid')

# To get different metric scores, and split data
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.utils import resample
from sklearn.metrics import confusion_matrix, accuracy_score, classification_report

# To get different tensorflow data
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Dropout,LeakyReLU, ReLU


# To supress warnings
import warnings

warnings.filterwarnings("ignore")

#checking tensorflow version
print(tf.__version__)

2.15.0

Loading the dataset

from google.colab import drive
drive.mount('/content/drive')

Mounted at /content/drive

# Loading the data into my colab notebook
data = pd.read_csv('/content/drive/My Drive/Deep Learning project/Churn.csv')

Step 2: Data Analysis & Visualization

Data Overview

# checking the headings of the data
data.head()

	RowNumber	CustomerId	Surname	CreditScore	Geography	Gender	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited
0	1	15634602	Hargrave	619	France	Female	42	2	0.00	1	1	1	101348.88	1
1	2	15647311	Hill	608	Spain	Female	41	1	83807.86	1	0	1	112542.58	0
2	3	15619304	Onio	502	France	Female	42	8	159660.80	3	1	0	113931.57	1
3	4	15701354	Boni	699	France	Female	39	1	0.00	2	0	0	93826.63	0
4	5	15737888	Mitchell	850	Spain	Female	43	2	125510.82	1	1	1	79084.10	0

Observation:

1. The dataframe contains 14 columns

2. Each columns contains information about customers showing their geographical locations, gender, ages, estimated salaries among others.

# checking the structure of the data
data.shape

(10000, 14)

Observation:

1. The dataframe has 14 columns and 10,000 rows

data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 10000 entries, 0 to 9999
Data columns (total 14 columns):
 #   Column           Non-Null Count  Dtype  
---  ------           --------------  -----  
 0   RowNumber        10000 non-null  int64  
 1   CustomerId       10000 non-null  int64  
 2   Surname          10000 non-null  object 
 3   CreditScore      10000 non-null  int64  
 4   Geography        10000 non-null  object 
 5   Gender           10000 non-null  object 
 6   Age              10000 non-null  int64  
 7   Tenure           10000 non-null  int64  
 8   Balance          10000 non-null  float64
 9   NumOfProducts    10000 non-null  int64  
 10  HasCrCard        10000 non-null  int64  
 11  IsActiveMember   10000 non-null  int64  
 12  EstimatedSalary  10000 non-null  float64
 13  Exited           10000 non-null  int64  
dtypes: float64(2), int64(9), object(3)
memory usage: 1.1+ MB

Observation:

1. The dataset have 14 columns and 10,000 observations in it
2. There are no columns that have missing values

data.describe().T

	count	mean	std	min	25%	50%	75%	max
RowNumber	10000.0	5.000500e+03	2886.895680	1.00	2500.75	5.000500e+03	7.500250e+03	10000.00
CustomerId	10000.0	1.569094e+07	71936.186123	15565701.00	15628528.25	1.569074e+07	1.575323e+07	15815690.00
CreditScore	10000.0	6.505288e+02	96.653299	350.00	584.00	6.520000e+02	7.180000e+02	850.00
Age	10000.0	3.892180e+01	10.487806	18.00	32.00	3.700000e+01	4.400000e+01	92.00
Tenure	10000.0	5.012800e+00	2.892174	0.00	3.00	5.000000e+00	7.000000e+00	10.00
Balance	10000.0	7.648589e+04	62397.405202	0.00	0.00	9.719854e+04	1.276442e+05	250898.09
NumOfProducts	10000.0	1.530200e+00	0.581654	1.00	1.00	1.000000e+00	2.000000e+00	4.00
HasCrCard	10000.0	7.055000e-01	0.455840	0.00	0.00	1.000000e+00	1.000000e+00	1.00
IsActiveMember	10000.0	5.151000e-01	0.499797	0.00	0.00	1.000000e+00	1.000000e+00	1.00
EstimatedSalary	10000.0	1.000902e+05	57510.492818	11.58	51002.11	1.001939e+05	1.493882e+05	199992.48
Exited	10000.0	2.037000e-01	0.402769	0.00	0.00	0.000000e+00	0.000000e+00	1.00

Observations:

1. Age has a large range of values i.e. 18 to 92.

2. Credit Score is between 350 to 850

3. Estimated Salary highest amount is USD199,992.48cents

4. More indept analysis is required.

#checking if there is any duplication in the dataset
data.duplicated().sum()

0

Observation:

1. There are no duplicated value in the dataset

data.isnull().sum()

RowNumber          0
CustomerId         0
Surname            0
CreditScore        0
Geography          0
Gender             0
Age                0
Tenure             0
Balance            0
NumOfProducts      0
HasCrCard          0
IsActiveMember     0
EstimatedSalary    0
Exited             0
dtype: int64

Observation:

1. There are no null values in the dataset

# Showing the unique values of each columns
data.nunique()

RowNumber          10000
CustomerId         10000
Surname             2932
CreditScore          460
Geography              3
Gender                 2
Age                   70
Tenure                11
Balance             6382
NumOfProducts          4
HasCrCard              2
IsActiveMember         2
EstimatedSalary     9999
Exited                 2
dtype: int64

Observation:

1. Customer Age has 70 unique values, balance 6382, credit score 460, tenure 11 e.t.c.
2. Only 1 of the columns is categotical items we would consider
3. There are 4 columns that contains binary category values in the dataset

# drop the irrelevant columns
data.drop(columns=["RowNumber", "CustomerId", "Surname"], inplace = True)

#checking the data after dropping irrelevant columns
data.head()

	CreditScore	Geography	Gender	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited
0	619	France	Female	42	2	0.00	1	1	1	101348.88	1
1	608	Spain	Female	41	1	83807.86	1	0	1	112542.58	0
2	502	France	Female	42	8	159660.80	3	1	0	113931.57	1
3	699	France	Female	39	1	0.00	2	0	0	93826.63	0
4	850	Spain	Female	43	2	125510.82	1	1	1	79084.10	0

Observation:

1. 3 columns have been dropped - Row number , Customer Id, Surname

Exploratory Data Analysis

• EDA is an important part of any project involving data.
• It is important to investigate and understand the data better before building a model with it.
• A few questions have been mentioned below which will help you approach the analysis in the right manner and generate insights from the data.
• A thorough analysis of the data, in addition to the questions mentioned below, should be done.

Questions:

1. What is the distribution of the credit score of customers? Are there any noticeable patterns or outliers in the distribution?
2. How many active members are there with the bank?
3. How are the different customer attributes correlated to each other?
4. Who is churning more when compared to males and females?
5. Customers from which geographical part are churning more?

# function to plot stacked bar chart

def stacked_barplot(data, predictor, target):
    """
    Print the category counts and plot a stacked bar chart

    data: dataframe
    predictor: independent variable
    target: target variable
    """
    count = data[predictor].nunique()
    sorter = data[target].value_counts().index[-1]
    tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(
        by=sorter, ascending=False
    )
    print(tab1)
    print("-" * 80)
    tab = pd.crosstab(data[predictor], data[target], normalize="index").sort_values(
        by=sorter, ascending=False
    )
    tab.plot(kind="bar", stacked=True, figsize=(count + 1, 3))
    plt.legend(
        loc="lower left", frameon=False,
    )
    plt.legend(loc="upper left", bbox_to_anchor=(1, 1))
    plt.show()

### Function to plot distributions

def distribution_plot_wrt_target(data, predictor, target):

    fig, axs = plt.subplots(2, 2, figsize=(12, 7))

    target_uniq = data[target].unique()

    axs[0, 0].set_title("Distribution of target for target variable=" + str(target_uniq[0]))
    sns.histplot(
        data=data[data[target] == target_uniq[0]],
        x=predictor,
        kde=True,
        ax=axs[0, 0],
        color="teal",
    )

    axs[0, 1].set_title("Distribution of target for target variable=" + str(target_uniq[1]))
    sns.histplot(
        data=data[data[target] == target_uniq[1]],
        x=predictor,
        kde=True,
        ax=axs[0, 1],
        color="orange",
    )

    axs[1, 0].set_title("Boxplot w.r.t target")
    sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette="gist_rainbow")

    axs[1, 1].set_title("Boxplot (without outliers) w.r.t target")
    sns.boxplot(
        data=data,
        x=target,
        y=predictor,
        ax=axs[1, 1],
        showfliers=False,
        palette="gist_rainbow",
    )

    plt.tight_layout()
    plt.show()

Geography

# check the value counts of "Geography" column.
data_geography = data['Geography'].value_counts()
data_geography

France     5014
Germany    2509
Spain      2477
Name: Geography, dtype: int64

# let's perform univarient eda on "Geography" column.
plt.figure(figsize=(10,6))
res=sns.barplot(x=data_geography, y=data_geography.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.title('Value Counts of Customers By Geographical Location',fontsize = 16, color='black')
plt.ylabel('Geographic Locations',fontsize = 16, color='black')
plt.xlabel('Counts', fontsize =16, color= 'black')
plt.show()

Observation:

1. Most of the Customers are from France, half of the customers are from this country which is 5,014
2. The remaining customers are from Germany and Spain, 2,509 and 2,477 respectively

Gender

# check the value counts of "Gender" column.
data_gender = data['Gender'].value_counts()
data_gender

Male      5457
Female    4543
Name: Gender, dtype: int64

# let's perform univarient eda on "Gender" column.
plt.figure(figsize=(8,6))
res=sns.barplot(x=data_gender, y=data_gender.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.title('Value Counts of Customers By Gender',fontsize = 16, color='black')
plt.ylabel('Gender',fontsize = 16, color='black')
plt.xlabel('Counts', fontsize =16, color= 'black')
plt.show()

Observation:

1. The distribution between the gender is close. Male customers are more, but not much different though.
2. Male is 5,457 and Female is 4,543
3. It is very interesting that there are more male customers

Number of Products

# check the value counts of "number of products" column.
data_nop = data['NumOfProducts'].value_counts()
data_nop

1    5084
2    4590
3     266
4      60
Name: NumOfProducts, dtype: int64

# let's perform univarient eda on "number of products" column.
plt.figure(figsize=(10,6))
plt.title('Value Counts of No of Products Used By The Customers',fontsize = 16, color='black')
plt.ylabel('Counts',fontsize = 16, color='black')
plt.xlabel('No of Products', fontsize =16, color= 'black')
res=sns.barplot(y=data_nop, x=data_nop.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.show();

Observation:

1. There are 4 number of products subscribed to by the customers
2. 5,084 customers have subscribed to product 1
3. 4,590 customers subscribed to product 2
4. 326 customers subcribed to products 3 and 4

Has Credit Card

# check the value counts of "HasCrCard" column.
data_crc = data['HasCrCard'].value_counts()
data_crc

1    7055
0    2945
Name: HasCrCard, dtype: int64

# let's perform univarient eda on "HasCrCard" column.
plt.figure(figsize=(8,6))
res=sns.barplot(y=data_crc, x=data_crc.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.ylabel('Counts of if Customers Have CC or Not',fontsize = 16, color='black')
plt.xlabel('Customer - Have CC or Not',fontsize = 16, color='black')
plt.title('Customers Who have Credit Cards or Not',fontsize = 16, color='black')
plt.show()

Observation:

1. More customers has credit cards - 7,055
2. While 29.45% of the customers did not have credit cards

Active or Not Active Customers

# check the value counts of "IsActiveMember" column.
data_iam = data['IsActiveMember'].value_counts()
data_iam

1    5151
0    4849
Name: IsActiveMember, dtype: int64

# let's perform univarient eda on "IsActiveMember" column.
plt.figure(figsize=(8,4))
res=sns.barplot(y=data_iam, x=data_iam.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.ylabel('Value Counts of Customer Status',fontsize = 16, color='black')
plt.xlabel('Active or Not Active',fontsize = 16, color='black')
plt.title('Status of Customers - Active or Not Active',fontsize = 16, color='black')
plt.show()

Observation:

1. 52% of the customers maintained active accounts with the bank while 48% had inactive accounts

Churned/Not Churned

# check the value counts of "Exited" target column.
data_exit = data['Exited'].value_counts()
data_exit

0    7963
1    2037
Name: Exited, dtype: int64

# # let's perform univarient eda on "Exited" target column.
plt.figure(figsize=(8,4))
res=sns.barplot(y=data_exit, x=data_exit.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.ylabel('No of Customers Who Churn or Not',fontsize = 16, color='black')
plt.xlabel('Churn/Not Churn',fontsize = 16, color='black')
plt.title('Customers Who Churned or Not',fontsize = 18, color='black')
plt.show()

Observation:

1. 80% of the customers churned the bank.
2. This is the area our neural network is goinf to focus on.

Age

distribution_plot_wrt_target(data, "Age", "Exited");

Observation:

1. The mode age is aroud 46. Most of the customers are young men around the age of 46
2. The Average age is arounf 36 years old, and the minimum age is around 18 yeras
3. There is outliers in the ages of the customers because some customer between the ages of 58 and 92 are present in the dataset
4. When the outliers in the age variable is treated, the mode age fell back to 38 years old.
5. The outliers in the age variable will not have any impact on our analysis because it is usually a natural course of life for ages to vary accordingly.

Balance

distribution_plot_wrt_target(data, "Balance", "Exited");

Observation

1. The Balance variable of the dataset show that most of the customers of the bank maintianed zero dollar for the period covered by the analysis.
2. This may be as a result of taking a cut-off period at the wrong time when the information were collated, or probably most of the bank accounts used in the analysis were salary accounts for workers and the information were collected after they may have withdrawn their money from their account or not. More question is needed to be asked in order to more fully understand why this is so.
3. The second most maintained account balance amount is around $125000. We need to find out why there is so much gap.
4. This independent variable is very important to our analysis.

Tenure

stacked_barplot(data, "Tenure", "Exited");

Exited     0     1    All
Tenure                   
All     7963  2037  10000
1        803   232   1035
3        796   213   1009
9        771   213    984
5        803   209   1012
4        786   203    989
2        847   201   1048
8        828   197   1025
6        771   196    967
7        851   177   1028
10       389   101    490
0        318    95    413
--------------------------------------------------------------------------------

Observation

1. Tenure can be described as the time period usually 12 months between the date of disbursal of a loan amount or credit card and so on, and the payment date of the last EMI
2. in the analysis, there are 11 tenures in the variable
3. The tunure of 2 years is the most commonly used tenure by the customers followed by tenure of 1 year.
4. It remains to be seen how this variable will affect our study.

Credit Score

distribution_plot_wrt_target(data, "CreditScore", "Exited");

Observation

1. The most credit score recorded and observed in the dataset is 650
2. 650 is also the average credit score in the dataset
3. But few customers have very low credit score, those that are regarded as very POOR credit score - Below 400 score

Estimated Salary

distribution_plot_wrt_target(data, "EstimatedSalary", "Exited");

Observation

1. Most of the Customers have estimated salsry of USD175000 followed by USD125000
2. The average estimated salary is USD100000
3. The percentage of customers whose estimated salary is above USD50000 constitute over 80% of the customers in the dataset
4. These variable show that most of the banks customers are middle class income earners and the bank should take advantage of this income bracket people.
5. This varaible wil surely have impact of whether a customer will churn or not churn. So it will be become more evident as this analysis is continued.

Pairplot of the dataset

# pairplot using Attrition flag, the dependent variable as emphasis
plt.figure(figsize=(15,7))
sns.pairplot(data, hue="Exited")
plt.show()
plt.savefig("output.jpg", bbox_inches='tight');

<Figure size 1500x700 with 0 Axes>

<Figure size 640x480 with 0 Axes>

Observation:

There are overlaps i.e. mostly no clear distinction in the distribution of most of the variables for people who have Churned and did not Churn.

1. No clear distinction in the distribution of variables for people who have churned and did not churn as the colorations of the pairplot are widely inter-mingled.
2. However, the following have lower amounts of churning customers;
   • Active or Not Active
   • Has Credit Card or Not
   • Number of Products
   • Tenure
3. The Following Variables showed evidence of contributing to if a customer will churn or not;
   • Balance
   • Estimated Salary
   • Age
   • Credit Score

This will guide my approacha and how to focus on my analysis.

Steps 3 to 11 are combined

Data Preprocessing

• Missing value treatment
• Feature engineering (if needed)
• Outlier detection and treatment (if needed)
• Preparing data for modeling
• Any other preprocessing steps (if needed)

Since our dataset contains categorical variables like “Gender” and “Geography,” we need to convert them into numerical format for the ANN model to process. We apply the one-hot encoding technique to transform these categorical columns into binary columns, representing different categories.

# use pandas dummies funtion for one hot encodeing
data = pd.get_dummies(data, columns=["Gender", "Geography"], drop_first=True)

#check the dataframe after the one hot encoding method is applied
data.head()

	CreditScore	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited	Gender_Male	Geography_Germany	Geography_Spain
0	619	42	2	0.00	1	1	1	101348.88	1	0	0	0
1	608	41	1	83807.86	1	0	1	112542.58	0	0	0	1
2	502	42	8	159660.80	3	1	0	113931.57	1	0	0	0
3	699	39	1	0.00	2	0	0	93826.63	0	0	0	0
4	850	43	2	125510.82	1	1	1	79084.10	0	0	0	1

data.sample(10)

	CreditScore	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited	Gender_Male	Geography_Germany	Geography_Spain
6666	588	32	3	109109.33	1	0	1	4993.94	0	1	0	1
7021	672	28	4	167268.98	1	1	1	169469.30	0	1	0	0
7454	732	46	0	0.00	2	1	1	184350.78	0	1	0	0
27	571	44	9	0.00	2	0	0	38433.35	0	1	0	0
7906	613	37	3	171653.17	1	0	1	5353.12	0	1	0	1
1925	804	24	3	0.00	2	1	0	173195.33	0	1	0	0
3823	627	31	8	128131.73	1	1	0	96131.47	0	1	0	0
8482	632	34	2	0.00	2	0	0	165385.55	0	0	0	0
5337	815	57	5	0.00	3	0	0	38941.44	1	0	0	1
5383	579	28	4	0.00	2	1	1	176925.69	0	1	0	0

.

Model Building Approach

Before training our ANN model, we need to preprocess the data by separating the features and the target variable. we split the dataset into training and testing sets. The training set is used to train the model on historical data, while the testing set is used to evaluate the model’s performance on unseen data. Typically, around 70–80% of the data is used for training, and the remaining 20–30% is used for testing.

Model Building

Model Building: Artificial Neural Network

#Starting the model building
from sklearn.model_selection import train_test_split
X = data.drop(columns=['Exited'])
y = data['Exited'].values

X_train_ann,X_test_ann,y_train_ann,y_test_ann=train_test_split(X,y,test_size=0.2,random_state=1)

# check the shape of X_train & X_test, y_train & y_test
print("X_train Shape : ", X_train_ann.shape)
print("X_test Shape : ", X_test_ann.shape)
print("y_train Shape : ", y_train_ann.shape)
print("y_test Shape : ", y_test_ann.shape)

X_train Shape :  (8000, 11)
X_test Shape :  (2000, 11)
y_train Shape :  (8000,)
y_test Shape :  (2000,)

# use sklearn standard scaler technique for standarized the input features values
scaler = StandardScaler()

X_train_ann = scaler.fit_transform(X_train_ann)
X_test_ann = scaler.transform(X_test_ann)

# check the X_train_scaled values
X_train_ann

array([[-0.23082038, -0.94449979, -0.70174202, ...,  0.91509065,
         1.71490137, -0.57273139],
       [-0.25150912, -0.94449979, -0.35520275, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [-0.3963303 ,  0.77498705,  0.33787579, ..., -1.09278791,
         1.71490137, -0.57273139],
       ...,
       [ 0.22433188,  0.58393295,  1.3774936 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 0.13123255,  0.01077067,  1.03095433, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 1.1656695 ,  0.29735181,  0.33787579, ...,  0.91509065,
         1.71490137, -0.57273139]])

# check the X_est_scaled values
X_test_ann

array([[-1.03768121,  0.77498705, -1.0482813 , ...,  0.91509065,
        -0.58312392, -0.57273139],
       [ 0.30708683, -0.46686456, -0.70174202, ...,  0.91509065,
        -0.58312392, -0.57273139],
       [-1.23422423,  0.29735181, -1.0482813 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       ...,
       [-0.86182692, -0.46686456,  1.72403288, ...,  0.91509065,
        -0.58312392,  1.74601919],
       [-0.30323097, -0.84897275, -1.0482813 , ..., -1.09278791,
         1.71490137, -0.57273139],
       [ 0.04847759,  1.25262228,  1.3774936 , ...,  0.91509065,
         1.71490137, -0.57273139]])

model_ann = Sequential()
model_ann.add(Dense(11, activation='sigmoid', input_dim = 11))
model_ann.add(Dense(11, activation='sigmoid'))
model_ann.add(Dense(1, activation='sigmoid'))

model_ann.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 11)                132       
                                                                 
 dense_1 (Dense)             (None, 11)                132       
                                                                 
 dense_2 (Dense)             (None, 1)                 12        
                                                                 
=================================================================
Total params: 276 (1.08 KB)
Trainable params: 276 (1.08 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

model_ann.compile(loss='binary_crossentropy', metrics=['accuracy'])

history_ann = model_ann.fit(X_train_ann, y_train_ann, epochs = 100, validation_split=0.2)

Epoch 1/100
200/200 [==============================] - 3s 4ms/step - loss: 0.5364 - accuracy: 0.7734 - val_loss: 0.4852 - val_accuracy: 0.7975
Epoch 2/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4747 - accuracy: 0.7972 - val_loss: 0.4676 - val_accuracy: 0.7975
Epoch 3/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4566 - accuracy: 0.7970 - val_loss: 0.4521 - val_accuracy: 0.7981
Epoch 4/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4418 - accuracy: 0.8014 - val_loss: 0.4421 - val_accuracy: 0.8050
Epoch 5/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4325 - accuracy: 0.8112 - val_loss: 0.4364 - val_accuracy: 0.8131
Epoch 6/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4279 - accuracy: 0.8141 - val_loss: 0.4341 - val_accuracy: 0.8087
Epoch 7/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4255 - accuracy: 0.8156 - val_loss: 0.4323 - val_accuracy: 0.8100
Epoch 8/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4237 - accuracy: 0.8166 - val_loss: 0.4311 - val_accuracy: 0.8106
Epoch 9/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4224 - accuracy: 0.8170 - val_loss: 0.4296 - val_accuracy: 0.8119
Epoch 10/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4210 - accuracy: 0.8169 - val_loss: 0.4289 - val_accuracy: 0.8156
Epoch 11/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4198 - accuracy: 0.8195 - val_loss: 0.4271 - val_accuracy: 0.8150
Epoch 12/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4184 - accuracy: 0.8205 - val_loss: 0.4248 - val_accuracy: 0.8175
Epoch 13/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4171 - accuracy: 0.8225 - val_loss: 0.4230 - val_accuracy: 0.8175
Epoch 14/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4154 - accuracy: 0.8241 - val_loss: 0.4211 - val_accuracy: 0.8213
Epoch 15/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4140 - accuracy: 0.8245 - val_loss: 0.4191 - val_accuracy: 0.8219
Epoch 16/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4122 - accuracy: 0.8263 - val_loss: 0.4176 - val_accuracy: 0.8206
Epoch 17/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4105 - accuracy: 0.8280 - val_loss: 0.4153 - val_accuracy: 0.8256
Epoch 18/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4088 - accuracy: 0.8308 - val_loss: 0.4133 - val_accuracy: 0.8256
Epoch 19/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4068 - accuracy: 0.8305 - val_loss: 0.4113 - val_accuracy: 0.8263
Epoch 20/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4048 - accuracy: 0.8322 - val_loss: 0.4088 - val_accuracy: 0.8256
Epoch 21/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4027 - accuracy: 0.8333 - val_loss: 0.4064 - val_accuracy: 0.8281
Epoch 22/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4006 - accuracy: 0.8322 - val_loss: 0.4044 - val_accuracy: 0.8306
Epoch 23/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3986 - accuracy: 0.8356 - val_loss: 0.4024 - val_accuracy: 0.8294
Epoch 24/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3965 - accuracy: 0.8384 - val_loss: 0.3998 - val_accuracy: 0.8306
Epoch 25/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3944 - accuracy: 0.8383 - val_loss: 0.3982 - val_accuracy: 0.8306
Epoch 26/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3923 - accuracy: 0.8392 - val_loss: 0.3963 - val_accuracy: 0.8338
Epoch 27/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3898 - accuracy: 0.8402 - val_loss: 0.3965 - val_accuracy: 0.8319
Epoch 28/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3881 - accuracy: 0.8436 - val_loss: 0.3918 - val_accuracy: 0.8344
Epoch 29/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3860 - accuracy: 0.8433 - val_loss: 0.3897 - val_accuracy: 0.8350
Epoch 30/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3840 - accuracy: 0.8433 - val_loss: 0.3874 - val_accuracy: 0.8363
Epoch 31/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3820 - accuracy: 0.8442 - val_loss: 0.3853 - val_accuracy: 0.8369
Epoch 32/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3798 - accuracy: 0.8466 - val_loss: 0.3834 - val_accuracy: 0.8369
Epoch 33/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3777 - accuracy: 0.8464 - val_loss: 0.3821 - val_accuracy: 0.8381
Epoch 34/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3761 - accuracy: 0.8481 - val_loss: 0.3795 - val_accuracy: 0.8381
Epoch 35/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3738 - accuracy: 0.8491 - val_loss: 0.3779 - val_accuracy: 0.8400
Epoch 36/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3722 - accuracy: 0.8498 - val_loss: 0.3757 - val_accuracy: 0.8381
Epoch 37/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3704 - accuracy: 0.8509 - val_loss: 0.3749 - val_accuracy: 0.8400
Epoch 38/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3689 - accuracy: 0.8506 - val_loss: 0.3740 - val_accuracy: 0.8388
Epoch 39/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3671 - accuracy: 0.8519 - val_loss: 0.3711 - val_accuracy: 0.8419
Epoch 40/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3655 - accuracy: 0.8533 - val_loss: 0.3697 - val_accuracy: 0.8406
Epoch 41/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3641 - accuracy: 0.8537 - val_loss: 0.3686 - val_accuracy: 0.8400
Epoch 42/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3626 - accuracy: 0.8536 - val_loss: 0.3683 - val_accuracy: 0.8400
Epoch 43/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3613 - accuracy: 0.8556 - val_loss: 0.3671 - val_accuracy: 0.8419
Epoch 44/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3599 - accuracy: 0.8552 - val_loss: 0.3644 - val_accuracy: 0.8438
Epoch 45/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3588 - accuracy: 0.8575 - val_loss: 0.3636 - val_accuracy: 0.8419
Epoch 46/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3571 - accuracy: 0.8561 - val_loss: 0.3624 - val_accuracy: 0.8438
Epoch 47/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3561 - accuracy: 0.8594 - val_loss: 0.3616 - val_accuracy: 0.8425
Epoch 48/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3552 - accuracy: 0.8583 - val_loss: 0.3599 - val_accuracy: 0.8456
Epoch 49/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3542 - accuracy: 0.8587 - val_loss: 0.3595 - val_accuracy: 0.8456
Epoch 50/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3530 - accuracy: 0.8606 - val_loss: 0.3585 - val_accuracy: 0.8462
Epoch 51/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3520 - accuracy: 0.8589 - val_loss: 0.3575 - val_accuracy: 0.8475
Epoch 52/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3512 - accuracy: 0.8592 - val_loss: 0.3568 - val_accuracy: 0.8450
Epoch 53/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3503 - accuracy: 0.8600 - val_loss: 0.3559 - val_accuracy: 0.8469
Epoch 54/100
200/200 [==============================] - 2s 8ms/step - loss: 0.3494 - accuracy: 0.8600 - val_loss: 0.3544 - val_accuracy: 0.8481
Epoch 55/100
200/200 [==============================] - 2s 9ms/step - loss: 0.3485 - accuracy: 0.8611 - val_loss: 0.3544 - val_accuracy: 0.8487
Epoch 56/100
200/200 [==============================] - 2s 8ms/step - loss: 0.3473 - accuracy: 0.8609 - val_loss: 0.3536 - val_accuracy: 0.8500
Epoch 57/100
200/200 [==============================] - 1s 7ms/step - loss: 0.3470 - accuracy: 0.8619 - val_loss: 0.3529 - val_accuracy: 0.8500
Epoch 58/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3456 - accuracy: 0.8609 - val_loss: 0.3523 - val_accuracy: 0.8512
Epoch 59/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3456 - accuracy: 0.8623 - val_loss: 0.3516 - val_accuracy: 0.8506
Epoch 60/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3447 - accuracy: 0.8623 - val_loss: 0.3509 - val_accuracy: 0.8519
Epoch 61/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3443 - accuracy: 0.8620 - val_loss: 0.3510 - val_accuracy: 0.8506
Epoch 62/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3434 - accuracy: 0.8627 - val_loss: 0.3507 - val_accuracy: 0.8506
Epoch 63/100
200/200 [==============================] - 2s 8ms/step - loss: 0.3429 - accuracy: 0.8636 - val_loss: 0.3497 - val_accuracy: 0.8537
Epoch 64/100
200/200 [==============================] - 1s 7ms/step - loss: 0.3424 - accuracy: 0.8630 - val_loss: 0.3489 - val_accuracy: 0.8544
Epoch 65/100
200/200 [==============================] - 2s 8ms/step - loss: 0.3419 - accuracy: 0.8630 - val_loss: 0.3485 - val_accuracy: 0.8537
Epoch 66/100
200/200 [==============================] - 2s 8ms/step - loss: 0.3410 - accuracy: 0.8630 - val_loss: 0.3481 - val_accuracy: 0.8519
Epoch 67/100
200/200 [==============================] - 2s 8ms/step - loss: 0.3407 - accuracy: 0.8636 - val_loss: 0.3477 - val_accuracy: 0.8537
Epoch 68/100
200/200 [==============================] - 2s 9ms/step - loss: 0.3395 - accuracy: 0.8653 - val_loss: 0.3487 - val_accuracy: 0.8525
Epoch 69/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3396 - accuracy: 0.8642 - val_loss: 0.3465 - val_accuracy: 0.8537
Epoch 70/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3390 - accuracy: 0.8637 - val_loss: 0.3474 - val_accuracy: 0.8537
Epoch 71/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3386 - accuracy: 0.8650 - val_loss: 0.3481 - val_accuracy: 0.8531
Epoch 72/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3380 - accuracy: 0.8630 - val_loss: 0.3456 - val_accuracy: 0.8537
Epoch 73/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3373 - accuracy: 0.8630 - val_loss: 0.3479 - val_accuracy: 0.8525
Epoch 74/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3373 - accuracy: 0.8639 - val_loss: 0.3453 - val_accuracy: 0.8544
Epoch 75/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3365 - accuracy: 0.8628 - val_loss: 0.3459 - val_accuracy: 0.8525
Epoch 76/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3362 - accuracy: 0.8645 - val_loss: 0.3446 - val_accuracy: 0.8525
Epoch 77/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3356 - accuracy: 0.8634 - val_loss: 0.3435 - val_accuracy: 0.8519
Epoch 78/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3351 - accuracy: 0.8644 - val_loss: 0.3456 - val_accuracy: 0.8500
Epoch 79/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3350 - accuracy: 0.8652 - val_loss: 0.3424 - val_accuracy: 0.8537
Epoch 80/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3345 - accuracy: 0.8642 - val_loss: 0.3421 - val_accuracy: 0.8519
Epoch 81/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3339 - accuracy: 0.8647 - val_loss: 0.3413 - val_accuracy: 0.8525
Epoch 82/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3338 - accuracy: 0.8645 - val_loss: 0.3429 - val_accuracy: 0.8556
Epoch 83/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3334 - accuracy: 0.8641 - val_loss: 0.3419 - val_accuracy: 0.8544
Epoch 84/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3330 - accuracy: 0.8662 - val_loss: 0.3403 - val_accuracy: 0.8512
Epoch 85/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3326 - accuracy: 0.8645 - val_loss: 0.3411 - val_accuracy: 0.8525
Epoch 86/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3322 - accuracy: 0.8656 - val_loss: 0.3406 - val_accuracy: 0.8531
Epoch 87/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3319 - accuracy: 0.8653 - val_loss: 0.3400 - val_accuracy: 0.8519
Epoch 88/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3316 - accuracy: 0.8655 - val_loss: 0.3405 - val_accuracy: 0.8537
Epoch 89/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3312 - accuracy: 0.8655 - val_loss: 0.3422 - val_accuracy: 0.8537
Epoch 90/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3305 - accuracy: 0.8647 - val_loss: 0.3424 - val_accuracy: 0.8519
Epoch 91/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3310 - accuracy: 0.8666 - val_loss: 0.3389 - val_accuracy: 0.8544
Epoch 92/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3301 - accuracy: 0.8662 - val_loss: 0.3432 - val_accuracy: 0.8512
Epoch 93/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3304 - accuracy: 0.8678 - val_loss: 0.3414 - val_accuracy: 0.8512
Epoch 94/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3301 - accuracy: 0.8677 - val_loss: 0.3397 - val_accuracy: 0.8525
Epoch 95/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3297 - accuracy: 0.8667 - val_loss: 0.3387 - val_accuracy: 0.8525
Epoch 96/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3297 - accuracy: 0.8650 - val_loss: 0.3386 - val_accuracy: 0.8531
Epoch 97/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3293 - accuracy: 0.8664 - val_loss: 0.3375 - val_accuracy: 0.8569
Epoch 98/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3292 - accuracy: 0.8659 - val_loss: 0.3382 - val_accuracy: 0.8519
Epoch 99/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3287 - accuracy: 0.8661 - val_loss: 0.3375 - val_accuracy: 0.8544
Epoch 100/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3289 - accuracy: 0.8655 - val_loss: 0.3383 - val_accuracy: 0.8544

#checking the loss visualization
plt.title('Visualization of Loss and Validation Loss of ANN Performance')
plt.plot(history_ann.history['loss'])
plt.plot(history_ann.history['val_loss']);

#checking the accuracy visualization
plt.title('Visualization of Accuracy and Validation Accuracy of ANN Performance')
plt.plot(history_ann.history['accuracy'])
plt.plot(history_ann.history['val_accuracy']);

y_log1 = model_ann.predict(X_test_ann)

63/63 [==============================] - 1s 5ms/step

y_pred_ann = np.where(y_log1 > 0.5, 1, 0)

accuracy_score(y_test_ann, y_pred_ann)

0.866

print(classification_report(y_test_ann, y_pred_ann))

              precision    recall  f1-score   support

           0       0.88      0.96      0.92      1585
           1       0.78      0.49      0.60       415

    accuracy                           0.87      2000
   macro avg       0.83      0.73      0.76      2000
weighted avg       0.86      0.87      0.85      2000

Observation:

The model generates a churn prediction score or probability of 86.60%, indicating the likelihood of the customer churning.

Model Building: Neural Network model with Adam Optimizer

X_train_adm,X_test_adm,y_train_adm,y_test_adm=train_test_split(X,y,test_size=0.2,random_state=1)

# check the shape of X_train & X_test, y_train & y_test
print("X_train Shape : ", X_train_adm.shape)
print("X_test Shape : ", X_test_adm.shape)
print("y_train Shape : ", y_train_adm.shape)
print("y_test Shape : ", y_test_adm.shape)

X_train Shape :  (8000, 11)
X_test Shape :  (2000, 11)
y_train Shape :  (8000,)
y_test Shape :  (2000,)

# use sklearn standard scaler technique for standarized the input features values
scaler = StandardScaler()

X_train_adm = scaler.fit_transform(X_train_adm)
X_test_adm = scaler.transform(X_test_adm)

# check the X_train_scaled values
X_train_adm

array([[-0.23082038, -0.94449979, -0.70174202, ...,  0.91509065,
         1.71490137, -0.57273139],
       [-0.25150912, -0.94449979, -0.35520275, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [-0.3963303 ,  0.77498705,  0.33787579, ..., -1.09278791,
         1.71490137, -0.57273139],
       ...,
       [ 0.22433188,  0.58393295,  1.3774936 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 0.13123255,  0.01077067,  1.03095433, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 1.1656695 ,  0.29735181,  0.33787579, ...,  0.91509065,
         1.71490137, -0.57273139]])

# check the X_train_scaled values
X_test_adm

array([[-1.03768121,  0.77498705, -1.0482813 , ...,  0.91509065,
        -0.58312392, -0.57273139],
       [ 0.30708683, -0.46686456, -0.70174202, ...,  0.91509065,
        -0.58312392, -0.57273139],
       [-1.23422423,  0.29735181, -1.0482813 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       ...,
       [-0.86182692, -0.46686456,  1.72403288, ...,  0.91509065,
        -0.58312392,  1.74601919],
       [-0.30323097, -0.84897275, -1.0482813 , ..., -1.09278791,
         1.71490137, -0.57273139],
       [ 0.04847759,  1.25262228,  1.3774936 , ...,  0.91509065,
         1.71490137, -0.57273139]])

model_adm = Sequential()
model_adm.add(Dense(11, activation='relu', input_dim = 11))
model_adm.add(Dense(11, activation='relu'))
model_adm.add(Dense(1, activation='sigmoid'))

model_adm.summary()

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_3 (Dense)             (None, 11)                132       
                                                                 
 dense_4 (Dense)             (None, 11)                132       
                                                                 
 dense_5 (Dense)             (None, 1)                 12        
                                                                 
=================================================================
Total params: 276 (1.08 KB)
Trainable params: 276 (1.08 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

model_adm.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy'])

history_adm =model_adm.fit(X_train_adm,y_train_adm,epochs=100,validation_split=0.2)

Epoch 1/100
200/200 [==============================] - 2s 4ms/step - loss: 0.5638 - accuracy: 0.7384 - val_loss: 0.4885 - val_accuracy: 0.8006
Epoch 2/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4607 - accuracy: 0.8017 - val_loss: 0.4491 - val_accuracy: 0.8037
Epoch 3/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4335 - accuracy: 0.8108 - val_loss: 0.4300 - val_accuracy: 0.8131
Epoch 4/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4151 - accuracy: 0.8197 - val_loss: 0.4139 - val_accuracy: 0.8206
Epoch 5/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3972 - accuracy: 0.8336 - val_loss: 0.3951 - val_accuracy: 0.8263
Epoch 6/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3802 - accuracy: 0.8448 - val_loss: 0.3782 - val_accuracy: 0.8356
Epoch 7/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3669 - accuracy: 0.8542 - val_loss: 0.3685 - val_accuracy: 0.8400
Epoch 8/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3588 - accuracy: 0.8577 - val_loss: 0.3616 - val_accuracy: 0.8381
Epoch 9/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3529 - accuracy: 0.8611 - val_loss: 0.3566 - val_accuracy: 0.8481
Epoch 10/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3491 - accuracy: 0.8623 - val_loss: 0.3546 - val_accuracy: 0.8469
Epoch 11/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3453 - accuracy: 0.8633 - val_loss: 0.3540 - val_accuracy: 0.8475
Epoch 12/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3436 - accuracy: 0.8634 - val_loss: 0.3508 - val_accuracy: 0.8519
Epoch 13/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3413 - accuracy: 0.8634 - val_loss: 0.3493 - val_accuracy: 0.8550
Epoch 14/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3397 - accuracy: 0.8633 - val_loss: 0.3483 - val_accuracy: 0.8525
Epoch 15/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3383 - accuracy: 0.8625 - val_loss: 0.3464 - val_accuracy: 0.8550
Epoch 16/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3371 - accuracy: 0.8631 - val_loss: 0.3458 - val_accuracy: 0.8569
Epoch 17/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3359 - accuracy: 0.8645 - val_loss: 0.3463 - val_accuracy: 0.8556
Epoch 18/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3352 - accuracy: 0.8628 - val_loss: 0.3448 - val_accuracy: 0.8594
Epoch 19/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3341 - accuracy: 0.8652 - val_loss: 0.3450 - val_accuracy: 0.8575
Epoch 20/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3333 - accuracy: 0.8658 - val_loss: 0.3447 - val_accuracy: 0.8575
Epoch 21/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3328 - accuracy: 0.8655 - val_loss: 0.3442 - val_accuracy: 0.8562
Epoch 22/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3325 - accuracy: 0.8639 - val_loss: 0.3451 - val_accuracy: 0.8562
Epoch 23/100
200/200 [==============================] - 2s 9ms/step - loss: 0.3318 - accuracy: 0.8652 - val_loss: 0.3460 - val_accuracy: 0.8562
Epoch 24/100
200/200 [==============================] - 2s 11ms/step - loss: 0.3318 - accuracy: 0.8656 - val_loss: 0.3450 - val_accuracy: 0.8587
Epoch 25/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3311 - accuracy: 0.8669 - val_loss: 0.3442 - val_accuracy: 0.8587
Epoch 26/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3308 - accuracy: 0.8648 - val_loss: 0.3462 - val_accuracy: 0.8556
Epoch 27/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3306 - accuracy: 0.8680 - val_loss: 0.3444 - val_accuracy: 0.8587
Epoch 28/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3301 - accuracy: 0.8655 - val_loss: 0.3449 - val_accuracy: 0.8550
Epoch 29/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3299 - accuracy: 0.8662 - val_loss: 0.3432 - val_accuracy: 0.8587
Epoch 30/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3294 - accuracy: 0.8661 - val_loss: 0.3437 - val_accuracy: 0.8550
Epoch 31/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3295 - accuracy: 0.8656 - val_loss: 0.3423 - val_accuracy: 0.8575
Epoch 32/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3285 - accuracy: 0.8661 - val_loss: 0.3447 - val_accuracy: 0.8531
Epoch 33/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3282 - accuracy: 0.8652 - val_loss: 0.3450 - val_accuracy: 0.8556
Epoch 34/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3286 - accuracy: 0.8652 - val_loss: 0.3427 - val_accuracy: 0.8581
Epoch 35/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3283 - accuracy: 0.8661 - val_loss: 0.3438 - val_accuracy: 0.8556
Epoch 36/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3277 - accuracy: 0.8653 - val_loss: 0.3428 - val_accuracy: 0.8575
Epoch 37/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3270 - accuracy: 0.8659 - val_loss: 0.3467 - val_accuracy: 0.8562
Epoch 38/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3282 - accuracy: 0.8666 - val_loss: 0.3422 - val_accuracy: 0.8562
Epoch 39/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3270 - accuracy: 0.8675 - val_loss: 0.3437 - val_accuracy: 0.8562
Epoch 40/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3271 - accuracy: 0.8662 - val_loss: 0.3424 - val_accuracy: 0.8600
Epoch 41/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3267 - accuracy: 0.8653 - val_loss: 0.3435 - val_accuracy: 0.8581
Epoch 42/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3271 - accuracy: 0.8650 - val_loss: 0.3441 - val_accuracy: 0.8556
Epoch 43/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3268 - accuracy: 0.8650 - val_loss: 0.3428 - val_accuracy: 0.8575
Epoch 44/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3263 - accuracy: 0.8653 - val_loss: 0.3443 - val_accuracy: 0.8581
Epoch 45/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3257 - accuracy: 0.8672 - val_loss: 0.3428 - val_accuracy: 0.8581
Epoch 46/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3260 - accuracy: 0.8672 - val_loss: 0.3434 - val_accuracy: 0.8556
Epoch 47/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3259 - accuracy: 0.8669 - val_loss: 0.3425 - val_accuracy: 0.8594
Epoch 48/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3257 - accuracy: 0.8667 - val_loss: 0.3433 - val_accuracy: 0.8575
Epoch 49/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3257 - accuracy: 0.8667 - val_loss: 0.3438 - val_accuracy: 0.8581
Epoch 50/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3251 - accuracy: 0.8670 - val_loss: 0.3442 - val_accuracy: 0.8544
Epoch 51/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3253 - accuracy: 0.8658 - val_loss: 0.3427 - val_accuracy: 0.8575
Epoch 52/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3252 - accuracy: 0.8669 - val_loss: 0.3437 - val_accuracy: 0.8562
Epoch 53/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3253 - accuracy: 0.8662 - val_loss: 0.3433 - val_accuracy: 0.8594
Epoch 54/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3248 - accuracy: 0.8681 - val_loss: 0.3435 - val_accuracy: 0.8569
Epoch 55/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3241 - accuracy: 0.8672 - val_loss: 0.3436 - val_accuracy: 0.8581
Epoch 56/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3247 - accuracy: 0.8666 - val_loss: 0.3452 - val_accuracy: 0.8562
Epoch 57/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3241 - accuracy: 0.8672 - val_loss: 0.3446 - val_accuracy: 0.8581
Epoch 58/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3240 - accuracy: 0.8661 - val_loss: 0.3436 - val_accuracy: 0.8569
Epoch 59/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3242 - accuracy: 0.8656 - val_loss: 0.3446 - val_accuracy: 0.8575
Epoch 60/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3240 - accuracy: 0.8670 - val_loss: 0.3443 - val_accuracy: 0.8587
Epoch 61/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3236 - accuracy: 0.8648 - val_loss: 0.3448 - val_accuracy: 0.8575
Epoch 62/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3236 - accuracy: 0.8662 - val_loss: 0.3445 - val_accuracy: 0.8537
Epoch 63/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3236 - accuracy: 0.8680 - val_loss: 0.3438 - val_accuracy: 0.8556
Epoch 64/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3232 - accuracy: 0.8664 - val_loss: 0.3454 - val_accuracy: 0.8544
Epoch 65/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3232 - accuracy: 0.8645 - val_loss: 0.3438 - val_accuracy: 0.8562
Epoch 66/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3228 - accuracy: 0.8680 - val_loss: 0.3463 - val_accuracy: 0.8544
Epoch 67/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3230 - accuracy: 0.8677 - val_loss: 0.3438 - val_accuracy: 0.8556
Epoch 68/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3224 - accuracy: 0.8667 - val_loss: 0.3458 - val_accuracy: 0.8550
Epoch 69/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3225 - accuracy: 0.8667 - val_loss: 0.3466 - val_accuracy: 0.8525
Epoch 70/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3229 - accuracy: 0.8652 - val_loss: 0.3480 - val_accuracy: 0.8506
Epoch 71/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3227 - accuracy: 0.8672 - val_loss: 0.3450 - val_accuracy: 0.8556
Epoch 72/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3217 - accuracy: 0.8680 - val_loss: 0.3470 - val_accuracy: 0.8531
Epoch 73/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3221 - accuracy: 0.8680 - val_loss: 0.3453 - val_accuracy: 0.8556
Epoch 74/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3218 - accuracy: 0.8670 - val_loss: 0.3475 - val_accuracy: 0.8569
Epoch 75/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3216 - accuracy: 0.8675 - val_loss: 0.3450 - val_accuracy: 0.8550
Epoch 76/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3218 - accuracy: 0.8680 - val_loss: 0.3464 - val_accuracy: 0.8544
Epoch 77/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3215 - accuracy: 0.8673 - val_loss: 0.3454 - val_accuracy: 0.8556
Epoch 78/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3213 - accuracy: 0.8681 - val_loss: 0.3452 - val_accuracy: 0.8562
Epoch 79/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3220 - accuracy: 0.8677 - val_loss: 0.3465 - val_accuracy: 0.8519
Epoch 80/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3208 - accuracy: 0.8691 - val_loss: 0.3481 - val_accuracy: 0.8525
Epoch 81/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3215 - accuracy: 0.8656 - val_loss: 0.3463 - val_accuracy: 0.8569
Epoch 82/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3211 - accuracy: 0.8672 - val_loss: 0.3458 - val_accuracy: 0.8544
Epoch 83/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3212 - accuracy: 0.8681 - val_loss: 0.3466 - val_accuracy: 0.8531
Epoch 84/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3211 - accuracy: 0.8669 - val_loss: 0.3457 - val_accuracy: 0.8550
Epoch 85/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3208 - accuracy: 0.8673 - val_loss: 0.3473 - val_accuracy: 0.8575
Epoch 86/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3205 - accuracy: 0.8692 - val_loss: 0.3472 - val_accuracy: 0.8506
Epoch 87/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3207 - accuracy: 0.8684 - val_loss: 0.3464 - val_accuracy: 0.8525
Epoch 88/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3205 - accuracy: 0.8680 - val_loss: 0.3467 - val_accuracy: 0.8531
Epoch 89/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3208 - accuracy: 0.8675 - val_loss: 0.3468 - val_accuracy: 0.8544
Epoch 90/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3208 - accuracy: 0.8669 - val_loss: 0.3463 - val_accuracy: 0.8537
Epoch 91/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3208 - accuracy: 0.8695 - val_loss: 0.3473 - val_accuracy: 0.8544
Epoch 92/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3202 - accuracy: 0.8673 - val_loss: 0.3459 - val_accuracy: 0.8544
Epoch 93/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3202 - accuracy: 0.8694 - val_loss: 0.3469 - val_accuracy: 0.8544
Epoch 94/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3202 - accuracy: 0.8673 - val_loss: 0.3470 - val_accuracy: 0.8531
Epoch 95/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3206 - accuracy: 0.8667 - val_loss: 0.3468 - val_accuracy: 0.8544
Epoch 96/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3201 - accuracy: 0.8692 - val_loss: 0.3465 - val_accuracy: 0.8569
Epoch 97/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3201 - accuracy: 0.8681 - val_loss: 0.3470 - val_accuracy: 0.8531
Epoch 98/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3200 - accuracy: 0.8670 - val_loss: 0.3466 - val_accuracy: 0.8544
Epoch 99/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3202 - accuracy: 0.8684 - val_loss: 0.3462 - val_accuracy: 0.8550
Epoch 100/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3196 - accuracy: 0.8677 - val_loss: 0.3458 - val_accuracy: 0.8531

#Checking the loss visualization
plt.title('Visualization of Loss and Valuation Loss Performance With Adam Optimizer')
plt.plot(history_adm.history['loss'])
plt.plot(history_adm.history['val_loss']);

#checking the accuracy visualization
plt.title('Visualization of Accuracy and Valuation Accuracy Performance With Adam Optimizer')
plt.plot(history_adm.history['accuracy'])
plt.plot(history_adm.history['val_accuracy']);

y_log_adm = model_adm.predict(X_test_adm)

63/63 [==============================] - 0s 2ms/step

y_pred_adm = np.where(y_log_adm > 0.5, 1, 0)

accuracy_score(y_test_adm, y_pred_adm)

0.868

print(classification_report(y_test_adm, y_pred_adm))

              precision    recall  f1-score   support

           0       0.88      0.96      0.92      1585
           1       0.77      0.51      0.62       415

    accuracy                           0.87      2000
   macro avg       0.83      0.74      0.77      2000
weighted avg       0.86      0.87      0.86      2000

Observation:

The model generates a churn prediction score or probability of 86.80%, indicating the likelihood of the customer churning.

Model Improvement: Neural Network model with Dropout Optimizer

X_train_dpo,X_test_dpo,y_train_dpo,y_test_dpo=train_test_split(X,y,test_size=0.2,random_state=1)

# check the shape of X_train & X_test, y_train & y_test
print("X_train Shape : ", X_train_dpo.shape)
print("X_test Shape : ", X_test_dpo.shape)
print("y_train Shape : ", y_train_dpo.shape)
print("y_test Shape : ", y_test_dpo.shape)

X_train Shape :  (8000, 11)
X_test Shape :  (2000, 11)
y_train Shape :  (8000,)
y_test Shape :  (2000,)

# use sklearn standard scaler technique for standarized the input features values
scaler = StandardScaler()

X_train_scaled_dpo = scaler.fit_transform(X_train_dpo)
X_test_scaled_dpo = scaler.transform(X_test_dpo)

X_train_scaled_dpo

array([[-0.23082038, -0.94449979, -0.70174202, ...,  0.91509065,
         1.71490137, -0.57273139],
       [-0.25150912, -0.94449979, -0.35520275, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [-0.3963303 ,  0.77498705,  0.33787579, ..., -1.09278791,
         1.71490137, -0.57273139],
       ...,
       [ 0.22433188,  0.58393295,  1.3774936 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 0.13123255,  0.01077067,  1.03095433, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 1.1656695 ,  0.29735181,  0.33787579, ...,  0.91509065,
         1.71490137, -0.57273139]])

X_test_scaled_dpo

array([[-1.03768121,  0.77498705, -1.0482813 , ...,  0.91509065,
        -0.58312392, -0.57273139],
       [ 0.30708683, -0.46686456, -0.70174202, ...,  0.91509065,
        -0.58312392, -0.57273139],
       [-1.23422423,  0.29735181, -1.0482813 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       ...,
       [-0.86182692, -0.46686456,  1.72403288, ...,  0.91509065,
        -0.58312392,  1.74601919],
       [-0.30323097, -0.84897275, -1.0482813 , ..., -1.09278791,
         1.71490137, -0.57273139],
       [ 0.04847759,  1.25262228,  1.3774936 , ...,  0.91509065,
         1.71490137, -0.57273139]])

# Define the model
model1 = Sequential()
model1.add(Dense(11, activation='relu', input_shape=(11,)))
model1.add(Dropout(0.5))
model1.add(Dense(11, activation='relu'))
model1.add(Dropout(0.2))
model1.add(Dense(1, activation='sigmoid'))

model1.summary()

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_6 (Dense)             (None, 11)                132       
                                                                 
 dropout (Dropout)           (None, 11)                0         
                                                                 
 dense_7 (Dense)             (None, 11)                132       
                                                                 
 dropout_1 (Dropout)         (None, 11)                0         
                                                                 
 dense_8 (Dense)             (None, 1)                 12        
                                                                 
=================================================================
Total params: 276 (1.08 KB)
Trainable params: 276 (1.08 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

# Compile the model
model1.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])

history_dpo = model1.fit(X_train_scaled_dpo, y_train_dpo, epochs = 100, validation_split=0.2)

Epoch 1/100
200/200 [==============================] - 3s 4ms/step - loss: 0.6110 - accuracy: 0.7322 - val_loss: 0.5129 - val_accuracy: 0.7975
Epoch 2/100
200/200 [==============================] - 1s 3ms/step - loss: 0.5265 - accuracy: 0.7917 - val_loss: 0.4775 - val_accuracy: 0.7975
Epoch 3/100
200/200 [==============================] - 1s 3ms/step - loss: 0.5010 - accuracy: 0.7958 - val_loss: 0.4622 - val_accuracy: 0.7975
Epoch 4/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4921 - accuracy: 0.7941 - val_loss: 0.4525 - val_accuracy: 0.7975
Epoch 5/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4689 - accuracy: 0.8011 - val_loss: 0.4431 - val_accuracy: 0.7994
Epoch 6/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4710 - accuracy: 0.8020 - val_loss: 0.4400 - val_accuracy: 0.8000
Epoch 7/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4619 - accuracy: 0.8000 - val_loss: 0.4363 - val_accuracy: 0.8000
Epoch 8/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4609 - accuracy: 0.8045 - val_loss: 0.4338 - val_accuracy: 0.8062
Epoch 9/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4579 - accuracy: 0.8052 - val_loss: 0.4340 - val_accuracy: 0.8025
Epoch 10/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4535 - accuracy: 0.8047 - val_loss: 0.4338 - val_accuracy: 0.8037
Epoch 11/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4517 - accuracy: 0.8052 - val_loss: 0.4339 - val_accuracy: 0.8056
Epoch 12/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4545 - accuracy: 0.8047 - val_loss: 0.4332 - val_accuracy: 0.8062
Epoch 13/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4495 - accuracy: 0.8084 - val_loss: 0.4310 - val_accuracy: 0.8044
Epoch 14/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4438 - accuracy: 0.8087 - val_loss: 0.4302 - val_accuracy: 0.8044
Epoch 15/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4480 - accuracy: 0.8109 - val_loss: 0.4316 - val_accuracy: 0.8044
Epoch 16/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4457 - accuracy: 0.8061 - val_loss: 0.4312 - val_accuracy: 0.8025
Epoch 17/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4459 - accuracy: 0.8081 - val_loss: 0.4312 - val_accuracy: 0.8025
Epoch 18/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4414 - accuracy: 0.8092 - val_loss: 0.4307 - val_accuracy: 0.8031
Epoch 19/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4445 - accuracy: 0.8078 - val_loss: 0.4306 - val_accuracy: 0.8025
Epoch 20/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4365 - accuracy: 0.8131 - val_loss: 0.4293 - val_accuracy: 0.8019
Epoch 21/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4374 - accuracy: 0.8125 - val_loss: 0.4276 - val_accuracy: 0.8031
Epoch 22/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4401 - accuracy: 0.8109 - val_loss: 0.4281 - val_accuracy: 0.8012
Epoch 23/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4373 - accuracy: 0.8142 - val_loss: 0.4262 - val_accuracy: 0.8050
Epoch 24/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4353 - accuracy: 0.8150 - val_loss: 0.4255 - val_accuracy: 0.8075
Epoch 25/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4362 - accuracy: 0.8159 - val_loss: 0.4237 - val_accuracy: 0.8100
Epoch 26/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4366 - accuracy: 0.8131 - val_loss: 0.4246 - val_accuracy: 0.8056
Epoch 27/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4344 - accuracy: 0.8158 - val_loss: 0.4243 - val_accuracy: 0.8081
Epoch 28/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4354 - accuracy: 0.8138 - val_loss: 0.4223 - val_accuracy: 0.8100
Epoch 29/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4332 - accuracy: 0.8161 - val_loss: 0.4211 - val_accuracy: 0.8131
Epoch 30/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4292 - accuracy: 0.8188 - val_loss: 0.4196 - val_accuracy: 0.8112
Epoch 31/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4334 - accuracy: 0.8186 - val_loss: 0.4177 - val_accuracy: 0.8125
Epoch 32/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4329 - accuracy: 0.8175 - val_loss: 0.4169 - val_accuracy: 0.8100
Epoch 33/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4284 - accuracy: 0.8223 - val_loss: 0.4160 - val_accuracy: 0.8131
Epoch 34/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4264 - accuracy: 0.8195 - val_loss: 0.4136 - val_accuracy: 0.8131
Epoch 35/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4244 - accuracy: 0.8252 - val_loss: 0.4119 - val_accuracy: 0.8138
Epoch 36/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4266 - accuracy: 0.8197 - val_loss: 0.4111 - val_accuracy: 0.8150
Epoch 37/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4220 - accuracy: 0.8225 - val_loss: 0.4031 - val_accuracy: 0.8219
Epoch 38/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4219 - accuracy: 0.8233 - val_loss: 0.4040 - val_accuracy: 0.8188
Epoch 39/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4227 - accuracy: 0.8241 - val_loss: 0.4031 - val_accuracy: 0.8213
Epoch 40/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4239 - accuracy: 0.8239 - val_loss: 0.4025 - val_accuracy: 0.8200
Epoch 41/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4145 - accuracy: 0.8270 - val_loss: 0.3982 - val_accuracy: 0.8250
Epoch 42/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4203 - accuracy: 0.8227 - val_loss: 0.3985 - val_accuracy: 0.8213
Epoch 43/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4222 - accuracy: 0.8261 - val_loss: 0.4007 - val_accuracy: 0.8231
Epoch 44/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4229 - accuracy: 0.8238 - val_loss: 0.3990 - val_accuracy: 0.8188
Epoch 45/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4168 - accuracy: 0.8261 - val_loss: 0.3962 - val_accuracy: 0.8219
Epoch 46/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4145 - accuracy: 0.8284 - val_loss: 0.3954 - val_accuracy: 0.8250
Epoch 47/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4167 - accuracy: 0.8255 - val_loss: 0.3961 - val_accuracy: 0.8275
Epoch 48/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4145 - accuracy: 0.8266 - val_loss: 0.3933 - val_accuracy: 0.8288
Epoch 49/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4130 - accuracy: 0.8295 - val_loss: 0.3950 - val_accuracy: 0.8263
Epoch 50/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4158 - accuracy: 0.8278 - val_loss: 0.3950 - val_accuracy: 0.8269
Epoch 51/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4156 - accuracy: 0.8264 - val_loss: 0.3957 - val_accuracy: 0.8306
Epoch 52/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4164 - accuracy: 0.8255 - val_loss: 0.3918 - val_accuracy: 0.8269
Epoch 53/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4155 - accuracy: 0.8273 - val_loss: 0.3948 - val_accuracy: 0.8288
Epoch 54/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4165 - accuracy: 0.8256 - val_loss: 0.3963 - val_accuracy: 0.8281
Epoch 55/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4203 - accuracy: 0.8245 - val_loss: 0.3990 - val_accuracy: 0.8256
Epoch 56/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4177 - accuracy: 0.8284 - val_loss: 0.3964 - val_accuracy: 0.8263
Epoch 57/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4200 - accuracy: 0.8245 - val_loss: 0.3962 - val_accuracy: 0.8231
Epoch 58/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4178 - accuracy: 0.8266 - val_loss: 0.3957 - val_accuracy: 0.8269
Epoch 59/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4184 - accuracy: 0.8256 - val_loss: 0.3969 - val_accuracy: 0.8263
Epoch 60/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4121 - accuracy: 0.8255 - val_loss: 0.3942 - val_accuracy: 0.8263
Epoch 61/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4153 - accuracy: 0.8244 - val_loss: 0.3955 - val_accuracy: 0.8263
Epoch 62/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4090 - accuracy: 0.8294 - val_loss: 0.3909 - val_accuracy: 0.8269
Epoch 63/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4115 - accuracy: 0.8288 - val_loss: 0.3942 - val_accuracy: 0.8294
Epoch 64/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4122 - accuracy: 0.8288 - val_loss: 0.3934 - val_accuracy: 0.8288
Epoch 65/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4134 - accuracy: 0.8306 - val_loss: 0.3945 - val_accuracy: 0.8319
Epoch 66/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4139 - accuracy: 0.8266 - val_loss: 0.3917 - val_accuracy: 0.8288
Epoch 67/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4199 - accuracy: 0.8261 - val_loss: 0.3959 - val_accuracy: 0.8269
Epoch 68/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4191 - accuracy: 0.8253 - val_loss: 0.3933 - val_accuracy: 0.8263
Epoch 69/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4115 - accuracy: 0.8273 - val_loss: 0.3917 - val_accuracy: 0.8281
Epoch 70/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4089 - accuracy: 0.8289 - val_loss: 0.3861 - val_accuracy: 0.8325
Epoch 71/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4177 - accuracy: 0.8244 - val_loss: 0.3924 - val_accuracy: 0.8256
Epoch 72/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4139 - accuracy: 0.8273 - val_loss: 0.3950 - val_accuracy: 0.8288
Epoch 73/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4114 - accuracy: 0.8289 - val_loss: 0.3933 - val_accuracy: 0.8275
Epoch 74/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4196 - accuracy: 0.8255 - val_loss: 0.3932 - val_accuracy: 0.8250
Epoch 75/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4210 - accuracy: 0.8217 - val_loss: 0.3969 - val_accuracy: 0.8219
Epoch 76/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4154 - accuracy: 0.8267 - val_loss: 0.3954 - val_accuracy: 0.8288
Epoch 77/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4147 - accuracy: 0.8256 - val_loss: 0.3927 - val_accuracy: 0.8219
Epoch 78/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4138 - accuracy: 0.8288 - val_loss: 0.3915 - val_accuracy: 0.8275
Epoch 79/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4133 - accuracy: 0.8291 - val_loss: 0.3909 - val_accuracy: 0.8281
Epoch 80/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4179 - accuracy: 0.8266 - val_loss: 0.3939 - val_accuracy: 0.8213
Epoch 81/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4138 - accuracy: 0.8272 - val_loss: 0.3921 - val_accuracy: 0.8244
Epoch 82/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4114 - accuracy: 0.8284 - val_loss: 0.3910 - val_accuracy: 0.8275
Epoch 83/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4109 - accuracy: 0.8273 - val_loss: 0.3915 - val_accuracy: 0.8281
Epoch 84/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4144 - accuracy: 0.8277 - val_loss: 0.3929 - val_accuracy: 0.8250
Epoch 85/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4140 - accuracy: 0.8267 - val_loss: 0.3935 - val_accuracy: 0.8275
Epoch 86/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4095 - accuracy: 0.8302 - val_loss: 0.3916 - val_accuracy: 0.8281
Epoch 87/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4141 - accuracy: 0.8272 - val_loss: 0.3913 - val_accuracy: 0.8250
Epoch 88/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4092 - accuracy: 0.8291 - val_loss: 0.3890 - val_accuracy: 0.8300
Epoch 89/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4167 - accuracy: 0.8261 - val_loss: 0.3927 - val_accuracy: 0.8244
Epoch 90/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4156 - accuracy: 0.8267 - val_loss: 0.3900 - val_accuracy: 0.8294
Epoch 91/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4080 - accuracy: 0.8297 - val_loss: 0.3894 - val_accuracy: 0.8288
Epoch 92/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4111 - accuracy: 0.8309 - val_loss: 0.3911 - val_accuracy: 0.8275
Epoch 93/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4113 - accuracy: 0.8258 - val_loss: 0.3900 - val_accuracy: 0.8300
Epoch 94/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4122 - accuracy: 0.8275 - val_loss: 0.3891 - val_accuracy: 0.8294
Epoch 95/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4161 - accuracy: 0.8247 - val_loss: 0.3922 - val_accuracy: 0.8263
Epoch 96/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4081 - accuracy: 0.8286 - val_loss: 0.3913 - val_accuracy: 0.8294
Epoch 97/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4058 - accuracy: 0.8292 - val_loss: 0.3879 - val_accuracy: 0.8344
Epoch 98/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4094 - accuracy: 0.8280 - val_loss: 0.3905 - val_accuracy: 0.8294
Epoch 99/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4056 - accuracy: 0.8297 - val_loss: 0.3880 - val_accuracy: 0.8325
Epoch 100/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4177 - accuracy: 0.8219 - val_loss: 0.3945 - val_accuracy: 0.8263

#checking loss visualization
plt.title('Visualization of Loss and Validation Loss Performance With DropOut Optimizer')
plt.plot(history_dpo.history['loss'])
plt.plot(history_dpo.history['val_loss']);

#checking the accuracy visualization
plt.title('Visualization of Accuracy and Validation Accuracy Performance With DropOut Optimizer')
plt.plot(history_dpo.history['accuracy'])
plt.plot(history_dpo.history['val_accuracy']);

y_log1c = model1.predict(X_test_scaled_dpo)

63/63 [==============================] - 0s 2ms/step

y_pred_dpo = np.where(y_log1c > 0.5, 1, 0)

accuracy_score(y_test_dpo, y_pred_dpo)

0.82

print(classification_report(y_test_dpo, y_pred_dpo))

              precision    recall  f1-score   support

           0       0.82      1.00      0.90      1585
           1       0.98      0.13      0.24       415

    accuracy                           0.82      2000
   macro avg       0.90      0.57      0.57      2000
weighted avg       0.85      0.82      0.76      2000

Observation:

The model generates a churn prediction score or probability of 82.00%, indicating the likelihood of the customer churning.

Model Improvement: Neural Network model with Hyperparameter Tuning

X_train_hpt,X_test_hpt,y_train_hpt,y_test_hpt=train_test_split(X,y,test_size=0.2,random_state=1)

# check the shape of X_train & X_test, y_train & y_test
print("X_train Shape : ", X_train_hpt.shape)
print("X_test Shape : ", X_test_hpt.shape)
print("y_train Shape : ", y_train_hpt.shape)
print("y_test Shape : ", y_test_hpt.shape)

X_train Shape :  (8000, 11)
X_test Shape :  (2000, 11)
y_train Shape :  (8000,)
y_test Shape :  (2000,)

# use sklearn standard scaler technique for standarized the input features values
scaler = StandardScaler()

X_train_scaled_hpt = scaler.fit_transform(X_train_hpt)
X_test_scaled_hpt = scaler.transform(X_test_hpt)

# check the X_train_scaled values
X_train_scaled_hpt

array([[-0.23082038, -0.94449979, -0.70174202, ...,  0.91509065,
         1.71490137, -0.57273139],
       [-0.25150912, -0.94449979, -0.35520275, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [-0.3963303 ,  0.77498705,  0.33787579, ..., -1.09278791,
         1.71490137, -0.57273139],
       ...,
       [ 0.22433188,  0.58393295,  1.3774936 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 0.13123255,  0.01077067,  1.03095433, ..., -1.09278791,
        -0.58312392, -0.57273139],
       [ 1.1656695 ,  0.29735181,  0.33787579, ...,  0.91509065,
         1.71490137, -0.57273139]])

X_test_scaled_hpt

array([[-1.03768121,  0.77498705, -1.0482813 , ...,  0.91509065,
        -0.58312392, -0.57273139],
       [ 0.30708683, -0.46686456, -0.70174202, ...,  0.91509065,
        -0.58312392, -0.57273139],
       [-1.23422423,  0.29735181, -1.0482813 , ..., -1.09278791,
        -0.58312392, -0.57273139],
       ...,
       [-0.86182692, -0.46686456,  1.72403288, ...,  0.91509065,
        -0.58312392,  1.74601919],
       [-0.30323097, -0.84897275, -1.0482813 , ..., -1.09278791,
         1.71490137, -0.57273139],
       [ 0.04847759,  1.25262228,  1.3774936 , ...,  0.91509065,
         1.71490137, -0.57273139]])

# Define the hyperparameter space
hyperparameters = {
    'activation': ['relu', 'sigmoid'],
    'learning_rate': [0.001,0.01,0.1,0.05]
}

# Create the neural network model
model_hpt = Sequential()
model_hpt.add(Dense(11, activation='relu', input_dim = 11))
model_hpt.add(Dense(11, activation='relu'))
model_hpt.add(Dense(1, activation='sigmoid'))

model_hpt.summary()

Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_12 (Dense)            (None, 11)                132       
                                                                 
 dense_13 (Dense)            (None, 11)                132       
                                                                 
 dense_14 (Dense)            (None, 1)                 12        
                                                                 
=================================================================
Total params: 276 (1.08 KB)
Trainable params: 276 (1.08 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

# Compile the model
model_hpt.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])

history_hpt = model_hpt.fit(X_train_scaled_hpt, y_train_hpt, epochs = 100, validation_split=0.2)

Epoch 1/100
200/200 [==============================] - 2s 6ms/step - loss: 0.5571 - accuracy: 0.7681 - val_loss: 0.4908 - val_accuracy: 0.7975
Epoch 2/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4607 - accuracy: 0.8042 - val_loss: 0.4447 - val_accuracy: 0.8112
Epoch 3/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4341 - accuracy: 0.8128 - val_loss: 0.4318 - val_accuracy: 0.8175
Epoch 4/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4232 - accuracy: 0.8155 - val_loss: 0.4226 - val_accuracy: 0.8181
Epoch 5/100
200/200 [==============================] - 1s 5ms/step - loss: 0.4154 - accuracy: 0.8216 - val_loss: 0.4162 - val_accuracy: 0.8181
Epoch 6/100
200/200 [==============================] - 1s 4ms/step - loss: 0.4102 - accuracy: 0.8247 - val_loss: 0.4113 - val_accuracy: 0.8231
Epoch 7/100
200/200 [==============================] - 1s 3ms/step - loss: 0.4036 - accuracy: 0.8281 - val_loss: 0.4029 - val_accuracy: 0.8275
Epoch 8/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3968 - accuracy: 0.8342 - val_loss: 0.3977 - val_accuracy: 0.8319
Epoch 9/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3905 - accuracy: 0.8364 - val_loss: 0.3881 - val_accuracy: 0.8325
Epoch 10/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3831 - accuracy: 0.8386 - val_loss: 0.3801 - val_accuracy: 0.8363
Epoch 11/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3747 - accuracy: 0.8483 - val_loss: 0.3710 - val_accuracy: 0.8444
Epoch 12/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3676 - accuracy: 0.8533 - val_loss: 0.3658 - val_accuracy: 0.8469
Epoch 13/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3623 - accuracy: 0.8555 - val_loss: 0.3604 - val_accuracy: 0.8494
Epoch 14/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3568 - accuracy: 0.8587 - val_loss: 0.3567 - val_accuracy: 0.8500
Epoch 15/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3529 - accuracy: 0.8602 - val_loss: 0.3539 - val_accuracy: 0.8544
Epoch 16/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3500 - accuracy: 0.8603 - val_loss: 0.3555 - val_accuracy: 0.8506
Epoch 17/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3471 - accuracy: 0.8595 - val_loss: 0.3520 - val_accuracy: 0.8531
Epoch 18/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3457 - accuracy: 0.8598 - val_loss: 0.3514 - val_accuracy: 0.8512
Epoch 19/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3443 - accuracy: 0.8587 - val_loss: 0.3495 - val_accuracy: 0.8531
Epoch 20/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3430 - accuracy: 0.8614 - val_loss: 0.3489 - val_accuracy: 0.8512
Epoch 21/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3418 - accuracy: 0.8605 - val_loss: 0.3488 - val_accuracy: 0.8556
Epoch 22/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3408 - accuracy: 0.8612 - val_loss: 0.3477 - val_accuracy: 0.8562
Epoch 23/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3404 - accuracy: 0.8620 - val_loss: 0.3479 - val_accuracy: 0.8525
Epoch 24/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3391 - accuracy: 0.8623 - val_loss: 0.3476 - val_accuracy: 0.8537
Epoch 25/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3386 - accuracy: 0.8614 - val_loss: 0.3461 - val_accuracy: 0.8569
Epoch 26/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3381 - accuracy: 0.8645 - val_loss: 0.3464 - val_accuracy: 0.8550
Epoch 27/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3372 - accuracy: 0.8614 - val_loss: 0.3489 - val_accuracy: 0.8506
Epoch 28/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3373 - accuracy: 0.8633 - val_loss: 0.3459 - val_accuracy: 0.8581
Epoch 29/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3366 - accuracy: 0.8633 - val_loss: 0.3461 - val_accuracy: 0.8550
Epoch 30/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3359 - accuracy: 0.8639 - val_loss: 0.3468 - val_accuracy: 0.8519
Epoch 31/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3351 - accuracy: 0.8650 - val_loss: 0.3455 - val_accuracy: 0.8575
Epoch 32/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3348 - accuracy: 0.8637 - val_loss: 0.3479 - val_accuracy: 0.8525
Epoch 33/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3347 - accuracy: 0.8627 - val_loss: 0.3471 - val_accuracy: 0.8519
Epoch 34/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3336 - accuracy: 0.8612 - val_loss: 0.3451 - val_accuracy: 0.8569
Epoch 35/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3328 - accuracy: 0.8645 - val_loss: 0.3512 - val_accuracy: 0.8494
Epoch 36/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3324 - accuracy: 0.8639 - val_loss: 0.3466 - val_accuracy: 0.8587
Epoch 37/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3322 - accuracy: 0.8634 - val_loss: 0.3475 - val_accuracy: 0.8519
Epoch 38/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3318 - accuracy: 0.8631 - val_loss: 0.3479 - val_accuracy: 0.8537
Epoch 39/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3319 - accuracy: 0.8644 - val_loss: 0.3462 - val_accuracy: 0.8587
Epoch 40/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3313 - accuracy: 0.8659 - val_loss: 0.3467 - val_accuracy: 0.8575
Epoch 41/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3311 - accuracy: 0.8636 - val_loss: 0.3456 - val_accuracy: 0.8575
Epoch 42/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3305 - accuracy: 0.8627 - val_loss: 0.3470 - val_accuracy: 0.8481
Epoch 43/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3303 - accuracy: 0.8628 - val_loss: 0.3482 - val_accuracy: 0.8487
Epoch 44/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3304 - accuracy: 0.8623 - val_loss: 0.3474 - val_accuracy: 0.8500
Epoch 45/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3296 - accuracy: 0.8628 - val_loss: 0.3461 - val_accuracy: 0.8569
Epoch 46/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3299 - accuracy: 0.8636 - val_loss: 0.3455 - val_accuracy: 0.8537
Epoch 47/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3288 - accuracy: 0.8644 - val_loss: 0.3463 - val_accuracy: 0.8562
Epoch 48/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3287 - accuracy: 0.8648 - val_loss: 0.3465 - val_accuracy: 0.8487
Epoch 49/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3284 - accuracy: 0.8628 - val_loss: 0.3469 - val_accuracy: 0.8487
Epoch 50/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3286 - accuracy: 0.8630 - val_loss: 0.3474 - val_accuracy: 0.8462
Epoch 51/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3276 - accuracy: 0.8614 - val_loss: 0.3521 - val_accuracy: 0.8481
Epoch 52/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3282 - accuracy: 0.8647 - val_loss: 0.3487 - val_accuracy: 0.8469
Epoch 53/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3280 - accuracy: 0.8648 - val_loss: 0.3468 - val_accuracy: 0.8487
Epoch 54/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3273 - accuracy: 0.8648 - val_loss: 0.3465 - val_accuracy: 0.8519
Epoch 55/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3275 - accuracy: 0.8634 - val_loss: 0.3478 - val_accuracy: 0.8512
Epoch 56/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3266 - accuracy: 0.8658 - val_loss: 0.3456 - val_accuracy: 0.8512
Epoch 57/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3260 - accuracy: 0.8647 - val_loss: 0.3511 - val_accuracy: 0.8475
Epoch 58/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3258 - accuracy: 0.8637 - val_loss: 0.3473 - val_accuracy: 0.8494
Epoch 59/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3262 - accuracy: 0.8648 - val_loss: 0.3492 - val_accuracy: 0.8438
Epoch 60/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3253 - accuracy: 0.8650 - val_loss: 0.3467 - val_accuracy: 0.8494
Epoch 61/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3252 - accuracy: 0.8642 - val_loss: 0.3479 - val_accuracy: 0.8550
Epoch 62/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3252 - accuracy: 0.8667 - val_loss: 0.3483 - val_accuracy: 0.8481
Epoch 63/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3257 - accuracy: 0.8639 - val_loss: 0.3471 - val_accuracy: 0.8506
Epoch 64/100
200/200 [==============================] - 1s 6ms/step - loss: 0.3249 - accuracy: 0.8655 - val_loss: 0.3482 - val_accuracy: 0.8519
Epoch 65/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3244 - accuracy: 0.8636 - val_loss: 0.3488 - val_accuracy: 0.8525
Epoch 66/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3241 - accuracy: 0.8639 - val_loss: 0.3485 - val_accuracy: 0.8550
Epoch 67/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3241 - accuracy: 0.8642 - val_loss: 0.3491 - val_accuracy: 0.8519
Epoch 68/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3238 - accuracy: 0.8645 - val_loss: 0.3500 - val_accuracy: 0.8512
Epoch 69/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3236 - accuracy: 0.8653 - val_loss: 0.3484 - val_accuracy: 0.8531
Epoch 70/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3239 - accuracy: 0.8659 - val_loss: 0.3504 - val_accuracy: 0.8494
Epoch 71/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3229 - accuracy: 0.8639 - val_loss: 0.3512 - val_accuracy: 0.8512
Epoch 72/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3229 - accuracy: 0.8636 - val_loss: 0.3493 - val_accuracy: 0.8512
Epoch 73/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3229 - accuracy: 0.8639 - val_loss: 0.3512 - val_accuracy: 0.8519
Epoch 74/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3226 - accuracy: 0.8641 - val_loss: 0.3505 - val_accuracy: 0.8519
Epoch 75/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3225 - accuracy: 0.8650 - val_loss: 0.3520 - val_accuracy: 0.8494
Epoch 76/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3222 - accuracy: 0.8648 - val_loss: 0.3515 - val_accuracy: 0.8512
Epoch 77/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3220 - accuracy: 0.8650 - val_loss: 0.3503 - val_accuracy: 0.8531
Epoch 78/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3222 - accuracy: 0.8673 - val_loss: 0.3534 - val_accuracy: 0.8462
Epoch 79/100
200/200 [==============================] - 2s 10ms/step - loss: 0.3212 - accuracy: 0.8653 - val_loss: 0.3525 - val_accuracy: 0.8494
Epoch 80/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3216 - accuracy: 0.8639 - val_loss: 0.3510 - val_accuracy: 0.8531
Epoch 81/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3209 - accuracy: 0.8662 - val_loss: 0.3545 - val_accuracy: 0.8500
Epoch 82/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3214 - accuracy: 0.8639 - val_loss: 0.3533 - val_accuracy: 0.8481
Epoch 83/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3206 - accuracy: 0.8652 - val_loss: 0.3500 - val_accuracy: 0.8537
Epoch 84/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3203 - accuracy: 0.8639 - val_loss: 0.3532 - val_accuracy: 0.8494
Epoch 85/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3203 - accuracy: 0.8664 - val_loss: 0.3503 - val_accuracy: 0.8575
Epoch 86/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3206 - accuracy: 0.8653 - val_loss: 0.3519 - val_accuracy: 0.8512
Epoch 87/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3203 - accuracy: 0.8670 - val_loss: 0.3516 - val_accuracy: 0.8519
Epoch 88/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3201 - accuracy: 0.8655 - val_loss: 0.3512 - val_accuracy: 0.8525
Epoch 89/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3202 - accuracy: 0.8653 - val_loss: 0.3537 - val_accuracy: 0.8494
Epoch 90/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3202 - accuracy: 0.8650 - val_loss: 0.3518 - val_accuracy: 0.8531
Epoch 91/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3194 - accuracy: 0.8655 - val_loss: 0.3536 - val_accuracy: 0.8494
Epoch 92/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3192 - accuracy: 0.8669 - val_loss: 0.3490 - val_accuracy: 0.8544
Epoch 93/100
200/200 [==============================] - 1s 3ms/step - loss: 0.3198 - accuracy: 0.8658 - val_loss: 0.3500 - val_accuracy: 0.8537
Epoch 94/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3192 - accuracy: 0.8672 - val_loss: 0.3509 - val_accuracy: 0.8537
Epoch 95/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3187 - accuracy: 0.8670 - val_loss: 0.3538 - val_accuracy: 0.8506
Epoch 96/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3192 - accuracy: 0.8673 - val_loss: 0.3512 - val_accuracy: 0.8556
Epoch 97/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3189 - accuracy: 0.8670 - val_loss: 0.3525 - val_accuracy: 0.8512
Epoch 98/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3190 - accuracy: 0.8670 - val_loss: 0.3526 - val_accuracy: 0.8550
Epoch 99/100
200/200 [==============================] - 1s 5ms/step - loss: 0.3190 - accuracy: 0.8684 - val_loss: 0.3521 - val_accuracy: 0.8537
Epoch 100/100
200/200 [==============================] - 1s 4ms/step - loss: 0.3186 - accuracy: 0.8675 - val_loss: 0.3507 - val_accuracy: 0.8550

#checking the loss visualization
plt.title('Visualization of Loss and Validation Loss Performance With Hyperparameter Tuning')
plt.plot(history_hpt.history['loss'])
plt.plot(history_hpt.history['val_loss']);

#checking the accuracy visualization
plt.title('Visualization of Accuracy and Validation Accuracy Performance With Hyperparameter Tuning')
plt.plot(history_hpt.history['accuracy'])
plt.plot(history_hpt.history['val_accuracy']);

y_log_hpt = model_hpt.predict(X_test_scaled_hpt)

63/63 [==============================] - 0s 2ms/step

y_pred_hpt = np.where(y_log_hpt > 0.5, 1, 0)

accuracy_score(y_test_hpt, y_pred_hpt)

0.8655

print(classification_report(y_test_hpt, y_pred_hpt))

              precision    recall  f1-score   support

           0       0.88      0.96      0.92      1585
           1       0.77      0.50      0.61       415

    accuracy                           0.87      2000
   macro avg       0.82      0.73      0.76      2000
weighted avg       0.86      0.87      0.85      2000

Observation:

The model generates a churn prediction score or probability of 86.55%, indicating the likelihood of the customer churning.

Model Improvement: Neural Network model with balanced data

Customer churn datasets often suffer from class imbalance, where the number of churners is significantly smaller than non-churners. To address this issue, we will balance the target column by upsampling the minority class. This ensures that the model does not get biased towards the majority class during training.

# Separate majority and minority classes
majority_class = data[data['Exited'] == 0]
minority_class = data[data['Exited'] == 1]

# Upsample the minority class
upsampled_minority = resample(minority_class,
                              replace=True,  # Sample with replacement
                              n_samples=len(majority_class),  # Match the number of majority class samples
                              random_state=42)  # Set random state for reproducibility

# Combine the upsampled minority class with the majority class
balanced_data = pd.concat([majority_class, upsampled_minority])

# check the target_balanced_data
balanced_data

	CreditScore	Age	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited	Gender_Male	Geography_Germany	Geography_Spain
1	608	41	1	83807.86	1	0	1	112542.58	0	0	0	1
3	699	39	1	0.00	2	0	0	93826.63	0	0	0	0
4	850	43	2	125510.82	1	1	1	79084.10	0	0	0	1
6	822	50	7	0.00	2	1	1	10062.80	0	1	0	0
8	501	44	4	142051.07	2	0	1	74940.50	0	1	0	0
...	...	...	...	...	...	...	...	...	...	...	...	...
5701	625	49	4	128504.76	1	1	0	126812.63	1	0	1	0
9337	466	47	5	102085.72	1	1	1	183536.24	1	0	1	0
4345	720	46	3	97042.60	1	1	1	133516.51	1	1	1	0
1085	803	42	5	0.00	1	1	0	196466.83	1	1	0	0
3694	608	33	4	0.00	1	1	0	79304.38	1	0	0	0

15926 rows × 12 columns

# check again the value counts of "Exited" target column after the balancing the data.
data_exit1 = balanced_data['Exited'].value_counts()
data_exit1

0    7963
1    7963
Name: Exited, dtype: int64

# # let's perform univarient eda on "Exited" target column.
plt.figure(figsize=(10,6))
res=sns.barplot(y=data_exit1, x=data_exit1.index)
res.set_yticklabels(res.get_ymajorticklabels(), fontsize = 16, color='black')
plt.ylabel('No of Customers Who Churn or Not',fontsize = 16, color='black')
plt.xlabel('Churn/Not Churn',fontsize = 16, color='black')
plt.title('Balanced Data - Customers Who Churned or Not',fontsize = 18, color='black')
plt.show()

# # split the feature in x variable and target varible in y variable
y = balanced_data['Exited']
X = balanced_data.drop('Exited',axis=1)

# use sklearn for train_test_split
X_train_bal, X_test_bal, y_train_bal, y_test_bal = train_test_split(X, y, test_size = 0.2, random_state = 1)

# check the shape of X_train & X_test, y_train & y_test
print("X_train Shape : ", X_train_bal.shape)
print("X_test Shape : ", X_test_bal.shape)
print("y_train Shape : ", y_train_bal.shape)
print("y_test Shape : ", y_test_bal.shape)

X_train Shape :  (12740, 11)
X_test Shape :  (3186, 11)
y_train Shape :  (12740,)
y_test Shape :  (3186,)

# use sklearn standard scaler technique for standarized the input features values
scaler = StandardScaler()

X_train_scaled_bal = scaler.fit_transform(X_train_bal)
X_test_scaled_bal = scaler.transform(X_test_bal)

# check the X_train_scaled values
X_train_scaled_bal

array([[-0.23415932,  0.17867584, -0.34133383, ...,  0.97442428,
        -0.6617577 , -0.54820692],
       [ 1.20116567,  0.83930236,  1.36239323, ..., -1.02624701,
        -0.6617577 , -0.54820692],
       [ 0.05290568, -0.76507633, -1.36357006, ...,  0.97442428,
        -0.6617577 , -0.54820692],
       ...,
       [ 0.9756146 ,  2.25493062, -1.02282465, ...,  0.97442428,
         1.51112711, -0.54820692],
       [ 0.47325086, -0.95382676,  1.70313864, ..., -1.02624701,
        -0.6617577 , -0.54820692],
       [ 0.30921371,  0.93367758, -1.36357006, ..., -1.02624701,
         1.51112711, -0.54820692]])

# check the X_est_scaled values
X_test_scaled_bal

array([[-0.37769182,  0.46180149,  0.340157  , ...,  0.97442428,
        -0.6617577 ,  1.82412874],
       [-0.72627074,  0.17867584, -0.68207924, ..., -1.02624701,
         1.51112711, -0.54820692],
       [ 0.28870907, -0.10444981, -1.02282465, ..., -1.02624701,
        -0.6617577 , -0.54820692],
       ...,
       [ 0.50400782, -0.85945155,  0.68090241, ...,  0.97442428,
        -0.6617577 , -0.54820692],
       [-0.04961753,  1.97180497,  1.36239323, ...,  0.97442428,
        -0.6617577 , -0.54820692],
       [ 0.47325086, -0.57632589,  0.68090241, ..., -1.02624701,
        -0.6617577 ,  1.82412874]])

model_bal = Sequential()
model_bal.add(Dense(11, activation='relu', input_dim = 11))
model_bal.add(Dense(11, activation='relu'))
model_bal.add(Dense(1, activation='sigmoid'))

model_bal.summary()

Model: "sequential_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_15 (Dense)            (None, 11)                132       
                                                                 
 dense_16 (Dense)            (None, 11)                132       
                                                                 
 dense_17 (Dense)            (None, 1)                 12        
                                                                 
=================================================================
Total params: 276 (1.08 KB)
Trainable params: 276 (1.08 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

model_bal.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

history_bal = model_bal.fit(X_train_scaled_bal, y_train_bal, epochs = 100, validation_split=0.2)

Epoch 1/100
319/319 [==============================] - 2s 4ms/step - loss: 0.6397 - accuracy: 0.6318 - val_loss: 0.5966 - val_accuracy: 0.6856
Epoch 2/100
319/319 [==============================] - 1s 4ms/step - loss: 0.5731 - accuracy: 0.7106 - val_loss: 0.5573 - val_accuracy: 0.7198
Epoch 3/100
319/319 [==============================] - 1s 4ms/step - loss: 0.5381 - accuracy: 0.7374 - val_loss: 0.5250 - val_accuracy: 0.7398
Epoch 4/100
319/319 [==============================] - 1s 4ms/step - loss: 0.5077 - accuracy: 0.7540 - val_loss: 0.4992 - val_accuracy: 0.7571
Epoch 5/100
319/319 [==============================] - 2s 7ms/step - loss: 0.4848 - accuracy: 0.7627 - val_loss: 0.4818 - val_accuracy: 0.7641
Epoch 6/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4712 - accuracy: 0.7707 - val_loss: 0.4696 - val_accuracy: 0.7728
Epoch 7/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4622 - accuracy: 0.7742 - val_loss: 0.4630 - val_accuracy: 0.7771
Epoch 8/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4557 - accuracy: 0.7769 - val_loss: 0.4576 - val_accuracy: 0.7779
Epoch 9/100
319/319 [==============================] - 1s 5ms/step - loss: 0.4517 - accuracy: 0.7795 - val_loss: 0.4543 - val_accuracy: 0.7783
Epoch 10/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4487 - accuracy: 0.7790 - val_loss: 0.4519 - val_accuracy: 0.7779
Epoch 11/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4460 - accuracy: 0.7850 - val_loss: 0.4507 - val_accuracy: 0.7786
Epoch 12/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4441 - accuracy: 0.7821 - val_loss: 0.4503 - val_accuracy: 0.7767
Epoch 13/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4420 - accuracy: 0.7868 - val_loss: 0.4483 - val_accuracy: 0.7767
Epoch 14/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4406 - accuracy: 0.7868 - val_loss: 0.4477 - val_accuracy: 0.7814
Epoch 15/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4395 - accuracy: 0.7886 - val_loss: 0.4469 - val_accuracy: 0.7834
Epoch 16/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4381 - accuracy: 0.7891 - val_loss: 0.4466 - val_accuracy: 0.7786
Epoch 17/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4374 - accuracy: 0.7910 - val_loss: 0.4468 - val_accuracy: 0.7830
Epoch 18/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4363 - accuracy: 0.7922 - val_loss: 0.4458 - val_accuracy: 0.7841
Epoch 19/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4352 - accuracy: 0.7920 - val_loss: 0.4453 - val_accuracy: 0.7802
Epoch 20/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4349 - accuracy: 0.7950 - val_loss: 0.4452 - val_accuracy: 0.7826
Epoch 21/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4330 - accuracy: 0.7935 - val_loss: 0.4453 - val_accuracy: 0.7845
Epoch 22/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4328 - accuracy: 0.7932 - val_loss: 0.4444 - val_accuracy: 0.7798
Epoch 23/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4311 - accuracy: 0.7980 - val_loss: 0.4455 - val_accuracy: 0.7865
Epoch 24/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4308 - accuracy: 0.7955 - val_loss: 0.4430 - val_accuracy: 0.7826
Epoch 25/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4298 - accuracy: 0.7957 - val_loss: 0.4429 - val_accuracy: 0.7838
Epoch 26/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4292 - accuracy: 0.7973 - val_loss: 0.4428 - val_accuracy: 0.7834
Epoch 27/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4281 - accuracy: 0.7980 - val_loss: 0.4420 - val_accuracy: 0.7873
Epoch 28/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4273 - accuracy: 0.7960 - val_loss: 0.4430 - val_accuracy: 0.7841
Epoch 29/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4272 - accuracy: 0.7984 - val_loss: 0.4420 - val_accuracy: 0.7845
Epoch 30/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4265 - accuracy: 0.8006 - val_loss: 0.4408 - val_accuracy: 0.7900
Epoch 31/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4260 - accuracy: 0.7996 - val_loss: 0.4396 - val_accuracy: 0.7861
Epoch 32/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4251 - accuracy: 0.8019 - val_loss: 0.4394 - val_accuracy: 0.7896
Epoch 33/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4247 - accuracy: 0.8020 - val_loss: 0.4411 - val_accuracy: 0.7885
Epoch 34/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4243 - accuracy: 0.8032 - val_loss: 0.4391 - val_accuracy: 0.7857
Epoch 35/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4238 - accuracy: 0.8023 - val_loss: 0.4396 - val_accuracy: 0.7877
Epoch 36/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4236 - accuracy: 0.8019 - val_loss: 0.4389 - val_accuracy: 0.7904
Epoch 37/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4231 - accuracy: 0.7999 - val_loss: 0.4394 - val_accuracy: 0.7916
Epoch 38/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4224 - accuracy: 0.8021 - val_loss: 0.4395 - val_accuracy: 0.7849
Epoch 39/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4225 - accuracy: 0.8010 - val_loss: 0.4371 - val_accuracy: 0.7924
Epoch 40/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4216 - accuracy: 0.8025 - val_loss: 0.4377 - val_accuracy: 0.7904
Epoch 41/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4214 - accuracy: 0.8000 - val_loss: 0.4395 - val_accuracy: 0.7936
Epoch 42/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4209 - accuracy: 0.8027 - val_loss: 0.4374 - val_accuracy: 0.7904
Epoch 43/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4203 - accuracy: 0.8030 - val_loss: 0.4372 - val_accuracy: 0.7932
Epoch 44/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4199 - accuracy: 0.8024 - val_loss: 0.4363 - val_accuracy: 0.7947
Epoch 45/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4192 - accuracy: 0.8037 - val_loss: 0.4360 - val_accuracy: 0.7943
Epoch 46/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4191 - accuracy: 0.8035 - val_loss: 0.4354 - val_accuracy: 0.7928
Epoch 47/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4188 - accuracy: 0.8033 - val_loss: 0.4351 - val_accuracy: 0.7963
Epoch 48/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4182 - accuracy: 0.8055 - val_loss: 0.4357 - val_accuracy: 0.7928
Epoch 49/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4185 - accuracy: 0.8039 - val_loss: 0.4357 - val_accuracy: 0.7967
Epoch 50/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4176 - accuracy: 0.8047 - val_loss: 0.4360 - val_accuracy: 0.7983
Epoch 51/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4172 - accuracy: 0.8046 - val_loss: 0.4352 - val_accuracy: 0.7971
Epoch 52/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4168 - accuracy: 0.8042 - val_loss: 0.4346 - val_accuracy: 0.7924
Epoch 53/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4167 - accuracy: 0.8054 - val_loss: 0.4345 - val_accuracy: 0.7951
Epoch 54/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4170 - accuracy: 0.8048 - val_loss: 0.4348 - val_accuracy: 0.7979
Epoch 55/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4167 - accuracy: 0.8047 - val_loss: 0.4367 - val_accuracy: 0.7916
Epoch 56/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4158 - accuracy: 0.8046 - val_loss: 0.4339 - val_accuracy: 0.7987
Epoch 57/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4156 - accuracy: 0.8057 - val_loss: 0.4342 - val_accuracy: 0.8022
Epoch 58/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4157 - accuracy: 0.8054 - val_loss: 0.4342 - val_accuracy: 0.7971
Epoch 59/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4154 - accuracy: 0.8067 - val_loss: 0.4332 - val_accuracy: 0.7971
Epoch 60/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4149 - accuracy: 0.8065 - val_loss: 0.4322 - val_accuracy: 0.7995
Epoch 61/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4151 - accuracy: 0.8070 - val_loss: 0.4348 - val_accuracy: 0.7955
Epoch 62/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4144 - accuracy: 0.8070 - val_loss: 0.4338 - val_accuracy: 0.7998
Epoch 63/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4140 - accuracy: 0.8072 - val_loss: 0.4337 - val_accuracy: 0.7987
Epoch 64/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4138 - accuracy: 0.8074 - val_loss: 0.4338 - val_accuracy: 0.7991
Epoch 65/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4136 - accuracy: 0.8065 - val_loss: 0.4341 - val_accuracy: 0.7983
Epoch 66/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4138 - accuracy: 0.8059 - val_loss: 0.4322 - val_accuracy: 0.8010
Epoch 67/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4132 - accuracy: 0.8078 - val_loss: 0.4339 - val_accuracy: 0.8014
Epoch 68/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4134 - accuracy: 0.8062 - val_loss: 0.4347 - val_accuracy: 0.7987
Epoch 69/100
319/319 [==============================] - 1s 5ms/step - loss: 0.4125 - accuracy: 0.8059 - val_loss: 0.4338 - val_accuracy: 0.8049
Epoch 70/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4124 - accuracy: 0.8080 - val_loss: 0.4348 - val_accuracy: 0.7959
Epoch 71/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4119 - accuracy: 0.8078 - val_loss: 0.4309 - val_accuracy: 0.8049
Epoch 72/100
319/319 [==============================] - 1s 5ms/step - loss: 0.4127 - accuracy: 0.8062 - val_loss: 0.4323 - val_accuracy: 0.8002
Epoch 73/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4120 - accuracy: 0.8094 - val_loss: 0.4340 - val_accuracy: 0.7967
Epoch 74/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4121 - accuracy: 0.8094 - val_loss: 0.4325 - val_accuracy: 0.8010
Epoch 75/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4115 - accuracy: 0.8075 - val_loss: 0.4315 - val_accuracy: 0.8014
Epoch 76/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4114 - accuracy: 0.8092 - val_loss: 0.4312 - val_accuracy: 0.8038
Epoch 77/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4111 - accuracy: 0.8088 - val_loss: 0.4314 - val_accuracy: 0.8030
Epoch 78/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4112 - accuracy: 0.8084 - val_loss: 0.4311 - val_accuracy: 0.8026
Epoch 79/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4113 - accuracy: 0.8088 - val_loss: 0.4319 - val_accuracy: 0.8042
Epoch 80/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4105 - accuracy: 0.8093 - val_loss: 0.4334 - val_accuracy: 0.7983
Epoch 81/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4110 - accuracy: 0.8111 - val_loss: 0.4301 - val_accuracy: 0.8018
Epoch 82/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4112 - accuracy: 0.8089 - val_loss: 0.4304 - val_accuracy: 0.8022
Epoch 83/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4101 - accuracy: 0.8101 - val_loss: 0.4329 - val_accuracy: 0.8002
Epoch 84/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4101 - accuracy: 0.8100 - val_loss: 0.4318 - val_accuracy: 0.8022
Epoch 85/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4105 - accuracy: 0.8107 - val_loss: 0.4305 - val_accuracy: 0.8010
Epoch 86/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4096 - accuracy: 0.8099 - val_loss: 0.4312 - val_accuracy: 0.8014
Epoch 87/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4099 - accuracy: 0.8110 - val_loss: 0.4309 - val_accuracy: 0.8014
Epoch 88/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4098 - accuracy: 0.8115 - val_loss: 0.4316 - val_accuracy: 0.7998
Epoch 89/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4092 - accuracy: 0.8091 - val_loss: 0.4298 - val_accuracy: 0.8030
Epoch 90/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4095 - accuracy: 0.8096 - val_loss: 0.4292 - val_accuracy: 0.8046
Epoch 91/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4090 - accuracy: 0.8126 - val_loss: 0.4304 - val_accuracy: 0.8022
Epoch 92/100
319/319 [==============================] - 2s 6ms/step - loss: 0.4090 - accuracy: 0.8106 - val_loss: 0.4318 - val_accuracy: 0.7983
Epoch 93/100
319/319 [==============================] - 2s 5ms/step - loss: 0.4089 - accuracy: 0.8106 - val_loss: 0.4301 - val_accuracy: 0.8014
Epoch 94/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4085 - accuracy: 0.8112 - val_loss: 0.4312 - val_accuracy: 0.7995
Epoch 95/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4082 - accuracy: 0.8119 - val_loss: 0.4297 - val_accuracy: 0.8042
Epoch 96/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4082 - accuracy: 0.8116 - val_loss: 0.4300 - val_accuracy: 0.8053
Epoch 97/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4081 - accuracy: 0.8099 - val_loss: 0.4295 - val_accuracy: 0.8097
Epoch 98/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4081 - accuracy: 0.8116 - val_loss: 0.4279 - val_accuracy: 0.8038
Epoch 99/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4078 - accuracy: 0.8120 - val_loss: 0.4301 - val_accuracy: 0.8042
Epoch 100/100
319/319 [==============================] - 1s 4ms/step - loss: 0.4080 - accuracy: 0.8116 - val_loss: 0.4288 - val_accuracy: 0.8018

#checking the loss visualization
plt.title('Visualization of Loss and Validation Loss Performance With Balanced Data')
plt.plot(history_bal.history['loss'])
plt.plot(history_bal.history['val_loss']);

#checking the accuracy visualization
plt.title('Visualization of Accuracy and Validation Accuracy Performance With Balanced Data')
plt.plot(history_bal.history['accuracy'])
plt.plot(history_bal.history['val_accuracy']);

y_log_bal = model_bal.predict(X_test_scaled_bal)

100/100 [==============================] - 0s 2ms/step

y_pred_bal = np.where(y_log_bal > 0.5, 1, 0)

accuracy_score(y_test_bal, y_pred_bal)

0.8010043942247332

print(classification_report(y_test_bal, y_pred_bal))

              precision    recall  f1-score   support

           0       0.79      0.81      0.80      1536
           1       0.82      0.80      0.81      1650

    accuracy                           0.80      3186
   macro avg       0.80      0.80      0.80      3186
weighted avg       0.80      0.80      0.80      3186

The model generates a churn prediction score or probability of 80.10%, indicating the likelihood of the customer churning.

Model Improvement: Neural Network model with Batch Normalization

# use sklearn for train_test_split
X_train_bn, X_test_bn, y_train_bn, y_test_bn = train_test_split(X, y, test_size = 0.2, random_state = 42)

# check the shape of X_train & X_test, y_train & y_test
print("X_train Shape : ", X_train_bn.shape)
print("X_test Shape : ", X_test_bn.shape)
print("y_train Shape : ", y_train_bn.shape)
print("y_test Shape : ", y_test_bn.shape)

X_train Shape :  (8000, 11)
X_test Shape :  (2000, 11)
y_train Shape :  (8000,)
y_test Shape :  (2000,)

# use sklearn standard scaler technique for standarized the input features values
scaler = StandardScaler()

X_train_scaled_bn = scaler.fit_transform(X_train_bn)
X_test_scaled_bn = scaler.transform(X_test_bn)

X_train_scaled_bn

array([[ 0.35649971, -0.6557859 ,  0.34567966, ...,  0.91324755,
        -0.57946723, -0.57638802],
       [-0.20389777,  0.29493847, -0.3483691 , ...,  0.91324755,
         1.72572313, -0.57638802],
       [-0.96147213, -1.41636539, -0.69539349, ...,  0.91324755,
        -0.57946723,  1.73494238],
       ...,
       [ 0.86500853, -0.08535128, -1.38944225, ..., -1.09499335,
        -0.57946723, -0.57638802],
       [ 0.15932282,  0.3900109 ,  1.03972843, ...,  0.91324755,
        -0.57946723, -0.57638802],
       [ 0.47065475,  1.15059039, -1.38944225, ...,  0.91324755,
         1.72572313, -0.57638802]])

X_test_scaled_bn

array([[-0.57749609, -0.6557859 , -0.69539349, ...,  0.91324755,
         1.72572313, -0.57638802],
       [-0.29729735,  0.3900109 , -1.38944225, ...,  0.91324755,
        -0.57946723, -0.57638802],
       [-0.52560743,  0.48508334, -0.3483691 , ..., -1.09499335,
        -0.57946723,  1.73494238],
       ...,
       [ 0.81311987,  0.77030065,  0.69270405, ..., -1.09499335,
        -0.57946723, -0.57638802],
       [ 0.41876609, -0.94100321, -0.3483691 , ...,  0.91324755,
        -0.57946723, -0.57638802],
       [-0.24540869,  0.00972116, -1.38944225, ...,  0.91324755,
         1.72572313, -0.57638802]])

model_bn = tf.keras.Sequential([
  tf.keras.layers.Dense(11, input_shape=(11,), activation='relu'),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(6, activation='relu'),
  tf.keras.layers.BatchNormalization(),
  tf.keras.layers.Dense(1, activation='sigmoid')
])

model_bn.summary()

Model: "sequential_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_9 (Dense)             (None, 11)                132       
                                                                 
 batch_normalization (Batch  (None, 11)                44        
 Normalization)                                                  
                                                                 
 dense_10 (Dense)            (None, 6)                 72        
                                                                 
 batch_normalization_1 (Bat  (None, 6)                 24        
 chNormalization)                                                
                                                                 
 dense_11 (Dense)            (None, 1)                 7         
                                                                 
=================================================================
Total params: 279 (1.09 KB)
Trainable params: 245 (980.00 Byte)
Non-trainable params: 34 (136.00 Byte)
_________________________________________________________________

model_bn.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

history_bn = model_bn.fit(X_train_scaled_bn, y_train_bn, epochs = 100, validation_split=0.25)

Epoch 1/100
188/188 [==============================] - 4s 6ms/step - loss: 0.6939 - accuracy: 0.6138 - val_loss: 0.5581 - val_accuracy: 0.7535
Epoch 2/100
188/188 [==============================] - 1s 6ms/step - loss: 0.4981 - accuracy: 0.7852 - val_loss: 0.4453 - val_accuracy: 0.8225
Epoch 3/100
188/188 [==============================] - 1s 7ms/step - loss: 0.4274 - accuracy: 0.8243 - val_loss: 0.4025 - val_accuracy: 0.8370
Epoch 4/100
188/188 [==============================] - 1s 8ms/step - loss: 0.4001 - accuracy: 0.8318 - val_loss: 0.3865 - val_accuracy: 0.8395
Epoch 5/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3880 - accuracy: 0.8355 - val_loss: 0.3781 - val_accuracy: 0.8365
Epoch 6/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3767 - accuracy: 0.8397 - val_loss: 0.3709 - val_accuracy: 0.8415
Epoch 7/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3747 - accuracy: 0.8402 - val_loss: 0.3665 - val_accuracy: 0.8460
Epoch 8/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3729 - accuracy: 0.8442 - val_loss: 0.3612 - val_accuracy: 0.8515
Epoch 9/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3669 - accuracy: 0.8465 - val_loss: 0.3593 - val_accuracy: 0.8485
Epoch 10/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3599 - accuracy: 0.8515 - val_loss: 0.3581 - val_accuracy: 0.8495
Epoch 11/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3576 - accuracy: 0.8498 - val_loss: 0.3569 - val_accuracy: 0.8495
Epoch 12/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3549 - accuracy: 0.8518 - val_loss: 0.3554 - val_accuracy: 0.8515
Epoch 13/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3559 - accuracy: 0.8525 - val_loss: 0.3550 - val_accuracy: 0.8495
Epoch 14/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3583 - accuracy: 0.8518 - val_loss: 0.3551 - val_accuracy: 0.8500
Epoch 15/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3509 - accuracy: 0.8548 - val_loss: 0.3535 - val_accuracy: 0.8505
Epoch 16/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3537 - accuracy: 0.8507 - val_loss: 0.3538 - val_accuracy: 0.8515
Epoch 17/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3513 - accuracy: 0.8555 - val_loss: 0.3527 - val_accuracy: 0.8525
Epoch 18/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3520 - accuracy: 0.8540 - val_loss: 0.3532 - val_accuracy: 0.8565
Epoch 19/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3484 - accuracy: 0.8538 - val_loss: 0.3524 - val_accuracy: 0.8565
Epoch 20/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3490 - accuracy: 0.8563 - val_loss: 0.3524 - val_accuracy: 0.8595
Epoch 21/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3476 - accuracy: 0.8602 - val_loss: 0.3518 - val_accuracy: 0.8570
Epoch 22/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3491 - accuracy: 0.8538 - val_loss: 0.3522 - val_accuracy: 0.8570
Epoch 23/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3435 - accuracy: 0.8545 - val_loss: 0.3527 - val_accuracy: 0.8545
Epoch 24/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3461 - accuracy: 0.8572 - val_loss: 0.3514 - val_accuracy: 0.8585
Epoch 25/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3427 - accuracy: 0.8565 - val_loss: 0.3511 - val_accuracy: 0.8570
Epoch 26/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3434 - accuracy: 0.8600 - val_loss: 0.3511 - val_accuracy: 0.8570
Epoch 27/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3438 - accuracy: 0.8587 - val_loss: 0.3504 - val_accuracy: 0.8595
Epoch 28/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3430 - accuracy: 0.8572 - val_loss: 0.3506 - val_accuracy: 0.8570
Epoch 29/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3409 - accuracy: 0.8565 - val_loss: 0.3501 - val_accuracy: 0.8570
Epoch 30/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3390 - accuracy: 0.8622 - val_loss: 0.3509 - val_accuracy: 0.8590
Epoch 31/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3368 - accuracy: 0.8593 - val_loss: 0.3511 - val_accuracy: 0.8590
Epoch 32/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3391 - accuracy: 0.8612 - val_loss: 0.3502 - val_accuracy: 0.8580
Epoch 33/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3375 - accuracy: 0.8603 - val_loss: 0.3506 - val_accuracy: 0.8600
Epoch 34/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3360 - accuracy: 0.8618 - val_loss: 0.3504 - val_accuracy: 0.8555
Epoch 35/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3331 - accuracy: 0.8622 - val_loss: 0.3497 - val_accuracy: 0.8595
Epoch 36/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3400 - accuracy: 0.8600 - val_loss: 0.3499 - val_accuracy: 0.8590
Epoch 37/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3400 - accuracy: 0.8573 - val_loss: 0.3510 - val_accuracy: 0.8585
Epoch 38/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3389 - accuracy: 0.8613 - val_loss: 0.3502 - val_accuracy: 0.8605
Epoch 39/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3352 - accuracy: 0.8640 - val_loss: 0.3501 - val_accuracy: 0.8580
Epoch 40/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3366 - accuracy: 0.8588 - val_loss: 0.3492 - val_accuracy: 0.8570
Epoch 41/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3352 - accuracy: 0.8595 - val_loss: 0.3492 - val_accuracy: 0.8600
Epoch 42/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3373 - accuracy: 0.8597 - val_loss: 0.3509 - val_accuracy: 0.8580
Epoch 43/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3363 - accuracy: 0.8588 - val_loss: 0.3502 - val_accuracy: 0.8560
Epoch 44/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3399 - accuracy: 0.8552 - val_loss: 0.3501 - val_accuracy: 0.8605
Epoch 45/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3321 - accuracy: 0.8617 - val_loss: 0.3493 - val_accuracy: 0.8585
Epoch 46/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3333 - accuracy: 0.8640 - val_loss: 0.3509 - val_accuracy: 0.8580
Epoch 47/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3366 - accuracy: 0.8635 - val_loss: 0.3511 - val_accuracy: 0.8570
Epoch 48/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3359 - accuracy: 0.8638 - val_loss: 0.3496 - val_accuracy: 0.8580
Epoch 49/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3360 - accuracy: 0.8612 - val_loss: 0.3518 - val_accuracy: 0.8600
Epoch 50/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3333 - accuracy: 0.8632 - val_loss: 0.3521 - val_accuracy: 0.8585
Epoch 51/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3379 - accuracy: 0.8598 - val_loss: 0.3518 - val_accuracy: 0.8585
Epoch 52/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3330 - accuracy: 0.8620 - val_loss: 0.3520 - val_accuracy: 0.8540
Epoch 53/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3304 - accuracy: 0.8673 - val_loss: 0.3510 - val_accuracy: 0.8605
Epoch 54/100
188/188 [==============================] - 2s 9ms/step - loss: 0.3306 - accuracy: 0.8635 - val_loss: 0.3507 - val_accuracy: 0.8585
Epoch 55/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3346 - accuracy: 0.8638 - val_loss: 0.3506 - val_accuracy: 0.8585
Epoch 56/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3324 - accuracy: 0.8627 - val_loss: 0.3514 - val_accuracy: 0.8585
Epoch 57/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3340 - accuracy: 0.8645 - val_loss: 0.3513 - val_accuracy: 0.8590
Epoch 58/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3409 - accuracy: 0.8580 - val_loss: 0.3520 - val_accuracy: 0.8590
Epoch 59/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3368 - accuracy: 0.8597 - val_loss: 0.3506 - val_accuracy: 0.8600
Epoch 60/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3332 - accuracy: 0.8640 - val_loss: 0.3523 - val_accuracy: 0.8585
Epoch 61/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3381 - accuracy: 0.8598 - val_loss: 0.3495 - val_accuracy: 0.8585
Epoch 62/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3329 - accuracy: 0.8622 - val_loss: 0.3517 - val_accuracy: 0.8590
Epoch 63/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3346 - accuracy: 0.8588 - val_loss: 0.3506 - val_accuracy: 0.8575
Epoch 64/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3362 - accuracy: 0.8613 - val_loss: 0.3506 - val_accuracy: 0.8590
Epoch 65/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3351 - accuracy: 0.8625 - val_loss: 0.3490 - val_accuracy: 0.8575
Epoch 66/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3335 - accuracy: 0.8640 - val_loss: 0.3515 - val_accuracy: 0.8550
Epoch 67/100
188/188 [==============================] - 2s 8ms/step - loss: 0.3365 - accuracy: 0.8585 - val_loss: 0.3519 - val_accuracy: 0.8560
Epoch 68/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3331 - accuracy: 0.8610 - val_loss: 0.3519 - val_accuracy: 0.8565
Epoch 69/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3287 - accuracy: 0.8635 - val_loss: 0.3502 - val_accuracy: 0.8595
Epoch 70/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3327 - accuracy: 0.8670 - val_loss: 0.3508 - val_accuracy: 0.8595
Epoch 71/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3331 - accuracy: 0.8588 - val_loss: 0.3509 - val_accuracy: 0.8560
Epoch 72/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3333 - accuracy: 0.8623 - val_loss: 0.3519 - val_accuracy: 0.8580
Epoch 73/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3342 - accuracy: 0.8623 - val_loss: 0.3515 - val_accuracy: 0.8595
Epoch 74/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3366 - accuracy: 0.8620 - val_loss: 0.3515 - val_accuracy: 0.8570
Epoch 75/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3344 - accuracy: 0.8613 - val_loss: 0.3517 - val_accuracy: 0.8570
Epoch 76/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3362 - accuracy: 0.8648 - val_loss: 0.3507 - val_accuracy: 0.8570
Epoch 77/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3329 - accuracy: 0.8622 - val_loss: 0.3534 - val_accuracy: 0.8510
Epoch 78/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3375 - accuracy: 0.8587 - val_loss: 0.3520 - val_accuracy: 0.8550
Epoch 79/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3331 - accuracy: 0.8610 - val_loss: 0.3516 - val_accuracy: 0.8550
Epoch 80/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3276 - accuracy: 0.8647 - val_loss: 0.3511 - val_accuracy: 0.8565
Epoch 81/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3343 - accuracy: 0.8592 - val_loss: 0.3506 - val_accuracy: 0.8575
Epoch 82/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3358 - accuracy: 0.8633 - val_loss: 0.3518 - val_accuracy: 0.8560
Epoch 83/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3321 - accuracy: 0.8628 - val_loss: 0.3505 - val_accuracy: 0.8580
Epoch 84/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3339 - accuracy: 0.8598 - val_loss: 0.3505 - val_accuracy: 0.8550
Epoch 85/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3322 - accuracy: 0.8633 - val_loss: 0.3495 - val_accuracy: 0.8615
Epoch 86/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3293 - accuracy: 0.8653 - val_loss: 0.3506 - val_accuracy: 0.8565
Epoch 87/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3349 - accuracy: 0.8610 - val_loss: 0.3491 - val_accuracy: 0.8575
Epoch 88/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3334 - accuracy: 0.8642 - val_loss: 0.3504 - val_accuracy: 0.8525
Epoch 89/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3331 - accuracy: 0.8632 - val_loss: 0.3487 - val_accuracy: 0.8560
Epoch 90/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3369 - accuracy: 0.8640 - val_loss: 0.3508 - val_accuracy: 0.8525
Epoch 91/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3337 - accuracy: 0.8618 - val_loss: 0.3522 - val_accuracy: 0.8540
Epoch 92/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3299 - accuracy: 0.8650 - val_loss: 0.3507 - val_accuracy: 0.8545
Epoch 93/100
188/188 [==============================] - 1s 7ms/step - loss: 0.3353 - accuracy: 0.8625 - val_loss: 0.3509 - val_accuracy: 0.8565
Epoch 94/100
188/188 [==============================] - 1s 8ms/step - loss: 0.3337 - accuracy: 0.8648 - val_loss: 0.3504 - val_accuracy: 0.8575
Epoch 95/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3303 - accuracy: 0.8677 - val_loss: 0.3517 - val_accuracy: 0.8525
Epoch 96/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3364 - accuracy: 0.8595 - val_loss: 0.3515 - val_accuracy: 0.8560
Epoch 97/100
188/188 [==============================] - 1s 5ms/step - loss: 0.3318 - accuracy: 0.8627 - val_loss: 0.3517 - val_accuracy: 0.8550
Epoch 98/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3327 - accuracy: 0.8603 - val_loss: 0.3518 - val_accuracy: 0.8565
Epoch 99/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3306 - accuracy: 0.8627 - val_loss: 0.3508 - val_accuracy: 0.8565
Epoch 100/100
188/188 [==============================] - 1s 6ms/step - loss: 0.3314 - accuracy: 0.8637 - val_loss: 0.3514 - val_accuracy: 0.8565

#checking the loss visualization
plt.title('Visualization of Loss and Validation Loss Performance With Batch Normalization')
plt.plot(history_bn.history['loss'])
plt.plot(history_bn.history['val_loss'])
plt.ylabel('loss')
plt.xlabel('epoch');

#checking the accuracy visualization
plt.title('Visualization of Accuracy and Validation Accuracy Performance With Batch Normalization')
plt.plot(history_bn.history['accuracy'])
plt.plot(history_bn.history['val_accuracy'])
plt.ylabel('accuracy')
plt.xlabel('epoch');

y_log_bn = model_bn.predict(X_test_scaled_bn)

63/63 [==============================] - 0s 2ms/step

y_pred_bn = np.where(y_log_bn > 0.5, 1, 0)

accuracy_score(y_test_bn, y_pred_bn)

0.8555

print(classification_report(y_test_bn, y_pred_bn))

              precision    recall  f1-score   support

           0       0.87      0.96      0.91      1607
           1       0.72      0.43      0.54       393

    accuracy                           0.86      2000
   macro avg       0.80      0.70      0.73      2000
weighted avg       0.84      0.86      0.84      2000

Observation:

The model generates a churn prediction score or probability of 85.55%, indicating the likelihood of the customer churning.

Step 12 and Final

# create a dataframe from the accuracy perfomance of the models
perf_accuracy = {
    "Artificial Neural Network Performance - Accuracy Score": 86.60,
    "Artificial Neural Network with Adam Optmizer Performance - Accuracy Score": 86.80,
    "Artificial Neural Network with DropOut Optimizer Performance - Accuracy Score": 82.00,
    "Artificial Neural Network with Hyperparameter Tuning Performance - Accuracy Score": 86.55,
    "Artificial Neural Network with Balanced Data Performance - Accuracy Score": 80.10,
    "Artificial Neural Network with Batch Normalization Performance - Accuracy Score": 85.55,
}
perf_accuracy = pd.DataFrame.from_dict(
    perf_accuracy, orient="index", columns=["Accuracy Scores"]
)
display(perf_accuracy.sort_values("Accuracy Scores"))

	Accuracy Scores
Artificial Neural Network with Balanced Data Performance - Accuracy Score	80.10
Artificial Neural Network with DropOut Optimizer Performance - Accuracy Score	82.00
Artificial Neural Network with Batch Normalization Performance - Accuracy Score	85.55
Artificial Neural Network with Hyperparameter Tuning Performance - Accuracy Score	86.55
Artificial Neural Network Performance - Accuracy Score	86.60
Artificial Neural Network with Adam Optmizer Performance - Accuracy Score	86.80

Observation:

All the 6 neural neteork used to analyse the data score 80% and above. However, 3 of them scored above 86% according to the results generated

There are 3 artificial neural network that scored 86% and above; these are

1. Artificial Neural Network Performance - Accuracy Score
2. Artificial Neural Network with Hyperparameter Tuning Performance - Accuracy Score
3. Artificial Neural Network with Adam Optmizer Performance - Accuracy Score

I will endeavor to check the features importance of these models in order to see how to understand the prediction performance in a real life situation.

Use the SHAP code to get the most importance feature of the model. I will use SHAP because our model analysis is a sequential analysis.

pip install shap

Collecting shap
  Downloading shap-0.44.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (533 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 533.5/533.5 kB 9.0 MB/s eta 0:00:00
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from shap) (1.23.5)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from shap) (1.11.4)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from shap) (1.2.2)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from shap) (1.5.3)
Requirement already satisfied: tqdm>=4.27.0 in /usr/local/lib/python3.10/dist-packages (from shap) (4.66.1)
Requirement already satisfied: packaging>20.9 in /usr/local/lib/python3.10/dist-packages (from shap) (23.2)
Collecting slicer==0.0.7 (from shap)
  Downloading slicer-0.0.7-py3-none-any.whl (14 kB)
Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from shap) (0.58.1)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.10/dist-packages (from shap) (2.2.1)
Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba->shap) (0.41.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->shap) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->shap) (2023.3.post1)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->shap) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->shap) (3.2.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->shap) (1.16.0)
Installing collected packages: slicer, shap
Successfully installed shap-0.44.0 slicer-0.0.7

Feature Imporatance - Model Optimized with Hyperparameter Tuning

import shap
from tensorflow.keras import Sequential

# load JS visualization code to notebook
shap.initjs()

# explain the model's predictions using SHAP
explainer = shap.DeepExplainer(model_hpt,data=X[:1500].values)
shap_values = explainer.shap_values(X.values)

# visualize the first prediction's explanation (use matplotlib=True to avoid Javascript)
#shap.force_plot(explainer.expected_value, shap_values[0,:], X.iloc[0,:])

shap.summary_plot(shap_values, X, plot_type="bar")

Feature Imporatance - Model - Neural Network

import shap
from tensorflow.keras import Sequential

# load JS visualization code to notebook
shap.initjs()

# explain the model's predictions using SHAP
explainer = shap.DeepExplainer(model_ann,data=X[:1500].values)
shap_values = explainer.shap_values(X.values)

# visualize the first prediction's explanation (use matplotlib=True to avoid Javascript)
#shap.force_plot(explainer.expected_value, shap_values[0,:], X.iloc[0,:])

shap.summary_plot(shap_values, X, plot_type="bar")

Feature Imporatance - Model Optimized with Adam Optimizer

import shap
from tensorflow.keras import Sequential

# load JS visualization code to notebook
shap.initjs()

# explain the model's predictions using SHAP
explainer = shap.DeepExplainer(model_adm,data=X[:1500].values)
shap_values = explainer.shap_values(X.values)

# visualize the first prediction's explanation (use matplotlib=True to avoid Javascript)
#shap.force_plot(explainer.expected_value, shap_values[0,:], X.iloc[0,:])

shap.summary_plot(shap_values, X, plot_type="bar")

Observation on the feature importance

All the 3 model showed that the following feature are the most important determining if a customers will churn or not churn;

1. Balance
2. Estimated Salary

Business Insights and Conclusions:

From the data visualizations that have been done so far, the following have been deduced:

1. Male bank customers churn the most with a percentage of 54% compared to females who have a percentage of 46%.

2. The dataset is about the customers in Europe - France, Germany and Spain.

3. Bank customers whose geographical location is France are more likely to churn thank those in Spain and Germany.

4. Customers who are using only 1 number of products are going to churn

5. Bank customers with Credit card no matter the types churn more with a percentage of 70% than those who did not.

6. Customers who are maintaining an active bank account are more likely to attrite than those who are inactive.

7. Bank customers whose age is around 46 are more likely to churn as far as the analysis of the dataset showed.

8. Customers whose estimated salary is around USD175000 per anum income level are more likely to churn than others.

   The ANN analysis done above showed that the estimated salary is the seconf most important feature of this neural network analysis

9. Bank customer who have had maintained $0 balance and from USD100000 to USD125000 are more likely to churn than others.

   This is the most important feature of this neural network analysis.

   All the 3 models above tuned showed that;

   1. those customers whose bank account balance were $0 between $100000 and $125000 are more likely to churn
   2. those whose bank balance is between USD100 and USD99500 and from USD130000 to USD200000 are less likely to churn.

10. Bank customers whose credit score of 650 are more likely to churn

11. Bank customers whose tenure is 2 years with the bank have a larger churn percentage than other categories

Recommendations:

1. It is very clear from the data analysis that the balance a customers maintained with the bank is an indicator if they will churn or not. Most of the customers prefer to not have any balance in their bank account, so the bank should consider doing one or all of the following;

   a. Giving interest on checking account balance

   b. Organizing a contest of leaving more balance in customers' bank account where a winner will emerge with prize.

   c. Marketing the use of saving account with an above the market intest rate incentive to the customers

2. The Bank should target any customers whose estimated salary is above USD50000 per anum. These customers are the most observed ones in the dataset as far as estimated salary is concerned because they constitute more than 80% of the customers base of the Bank. The bank should organize an aggressive marketting that will focus on these customers in order to showcase all the offers available. Offers like long moratorium credit cards, expansive cashback deals, expansive mileage deals, competitive APR, Credit card back designs et cetera.

3. Bank should also offer credit limit increase for the customers who are regularly using their credit cards especially those customers whose estinated salary is above USD50000. This would definitely increase the credit card spends/transaction amounts and will discorage churning rate.

4. 0% interest EMI (equated monthly installment) on credit card is also another offer that can be provided to customers to encourage them to buy products of higher cost with their credit card, and convert the expenditure to EMI. This will increase the total transaction amount as well as transaction counts of the customers in a given cut-off period. Further, this would have a simultaneous effect on the rovilving balance as well.

5. Along with the available types of cards, bank can introduce credit cards specifically directed to influence more online shopping (with certain percentage %ge cashback offers) or online food ordering on Doordash, Ubereats, Grubhub etc. This would geometrically increase the card usage per period and would make it nearly difficult for the customers to churn the card.

6. With our model, it is expediently possible to predict which customers are likely to churn. Also according to the predicted probability, so arrangements should be made for the marketting department to reach through different advertising schemes to at least top 50-80% customers in order to show them new credit card offers, increased credit limit, newly opened cashback regimes et cetera. This can be used as a pilot program to try retain these customers and afterwards take it to everybody else.

7. The bank shouls also consider looking into those Customers whose credit score are very poor below 400. Free information service should be provided on what and how these customers can works towards improving their credit score. If successful, these customers will prove loyal to the Bank and thus will not likely churn. Moreover, low credit limit credit card should be provided for these customers to help them on their road to improving their credit score.

8. Though, its observed that male customers are more likely to churn more than female ones, attention should be given to other demographical information, for example the geographical locations of these customers if its has any bearing on the behavioral trend or patterns of men. The bank should organize more study in this area because this study did not get any serious pattern from geographical location. A new study focusing on geographical location make be required.

9. As regards tenure, the Bank should consider giving credit cards with tenure ranging from 18 months to 24 months with zero dollar APR to all the customers irrespective of their balance amount or their estimated salary.

!pip install nbconvert

Requirement already satisfied: nbconvert in /usr/local/lib/python3.10/dist-packages (6.5.4)
Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from nbconvert) (4.9.3)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (4.11.2)
Requirement already satisfied: bleach in /usr/local/lib/python3.10/dist-packages (from nbconvert) (6.1.0)
Requirement already satisfied: defusedxml in /usr/local/lib/python3.10/dist-packages (from nbconvert) (0.7.1)
Requirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (0.4)
Requirement already satisfied: jinja2>=3.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (3.1.2)
Requirement already satisfied: jupyter-core>=4.7 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (5.5.1)
Requirement already satisfied: jupyterlab-pygments in /usr/local/lib/python3.10/dist-packages (from nbconvert) (0.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (2.1.3)
Requirement already satisfied: mistune<2,>=0.8.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (0.8.4)
Requirement already satisfied: nbclient>=0.5.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (0.9.0)
Requirement already satisfied: nbformat>=5.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (5.9.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from nbconvert) (23.2)
Requirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (1.5.0)
Requirement already satisfied: pygments>=2.4.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (2.16.1)
Requirement already satisfied: tinycss2 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (1.2.1)
Requirement already satisfied: traitlets>=5.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert) (5.7.1)
Requirement already satisfied: platformdirs>=2.5 in /usr/local/lib/python3.10/dist-packages (from jupyter-core>=4.7->nbconvert) (4.1.0)
Requirement already satisfied: jupyter-client>=6.1.12 in /usr/local/lib/python3.10/dist-packages (from nbclient>=0.5.0->nbconvert) (6.1.12)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.10/dist-packages (from nbformat>=5.1->nbconvert) (2.19.0)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.10/dist-packages (from nbformat>=5.1->nbconvert) (4.19.2)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->nbconvert) (2.5)
Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from bleach->nbconvert) (1.16.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from bleach->nbconvert) (0.5.1)
Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (23.1.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (2023.11.2)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (0.32.0)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=5.1->nbconvert) (0.15.2)
Requirement already satisfied: pyzmq>=13 in /usr/local/lib/python3.10/dist-packages (from jupyter-client>=6.1.12->nbclient>=0.5.0->nbconvert) (23.2.1)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.10/dist-packages (from jupyter-client>=6.1.12->nbclient>=0.5.0->nbconvert) (2.8.2)
Requirement already satisfied: tornado>=4.1 in /usr/local/lib/python3.10/dist-packages (from jupyter-client>=6.1.12->nbclient>=0.5.0->nbconvert) (6.3.2)

%%shell
jupyter nbconvert --to html '/content/drive/My Drive/Deep Learning project/Samson_Akomolafe_Deep_Learning_Fullcode_Project_4a.ipynb'

[NbConvertApp] Converting notebook /content/drive/My Drive/Deep Learning project/Samson_Akomolafe_Deep_Learning_Fullcode_Project_4a.ipynb to html
[NbConvertApp] Writing 5512353 bytes to /content/drive/My Drive/Deep Learning project/Samson_Akomolafe_Deep_Learning_Fullcode_Project_4a.html

*

---